








Psychometrika 


CONTENTS 


THE VARIMAX CRITERION FOR ANALYTIC ROTATION IN 
I I ig ee aye cee wes eae 187 
Henry F. Kaiser 


POWER FUNCTION CHARTS FOR SPECIFICATION OF 
SAMPLE SIZE IN ANALYSIS OF VARIANCE. .... 201 
LEONARD S. FELDT AND MoHarrRAM W. MAuHMoupD 


A COMPARATIVE STUDY OF THREE METHODS OF 
I AR CEE SS rc a RO ot EE 211 
BENJAMIN FRUCHTER AND Epwin Novak 
ANALYSIS OF VARIANCE FOR CORRELATED OBSERVATIONS 223 
Raymonp O. Co..ieEr, JR. 
THURSTONE’S ANALYTICAL METHOD FOR SIMPLE 
STRUCTURE AND A MASS MODIFICATION THEREOF 237 
Rosert R. SoKAL 
ATTENUATION AND INTERACTION ............ 259 
Quinn McNEMAR 
THE KUDER-RICHARDSON FORMULA (21) AS A SPLIT-HALF 
COEFFICIENT, AND REMARKS ON ITS BASIC 
I, ee gg a ko WV we at 267 
SAMUEL B. LYERLY 
THE AVERAGE SPEARMAN RANK CRITERION CORRELA- 
TION WHEN TIES ARE PRESENT ......... 271 
Epwarpb E. CuRETON 
NOTE ON “EFFICIENT ESTIMATION AND LOCAL 
IDENTIFICATION IN LATENT CLASS ANALYSIS” . 273 
Ricuarp B. McHucu 
BOOK REVIEWS 
Henry QuastieR (Editor). Information Theory in Psychology . 275 
Review by Joun B. Carroiu 





Wixtu1aAm G. CocHraN AND GerRTRUDE M. Cox. Experimental 
SN, I IN orcs k hea es 0s we ale 277 
Review by J. R. WirrENBoRN 








VOLUME TWENTY-THREE SEPTEMBER 1958 NUMBER 3 








PSYCHOMETRIKA—VOL. 23, NO. 3 
SEPTEMBER, 1958 ‘ 


THE VARIMAX CRITERION FOR ANALYTIC ROTATION IN 
FACTOR ANALYSIS* 


Henry F. Katser 
UNIVERSITY OF ILLINOIS 


An analytic criterion for rotation is defined. The scientific advantage 
of analytic criteria over subjective (graphical) rotational procedures is dis- 
cussed. Carroll’s criterion and the quartimax criterion are briefly reviewed; 
the varimax criterion is outlined in detail and contrasted both logically and 
numerically with the quartimax criterion. It is shown that the normal varimax 
solution probably coincides closely to the application of the principle of simple 
structure. However, it is proposed that the ultimate criterion of a rotational 
procedure is factorial invariance, not simple structure—although the two 
notions appear to be highly related. The normal varimax criterion is shown 
to be a two-dimensional generalization of the classic Spearman case, i.e., it 
shows perfect factorial invariance for two pure clusters. An example is 
given of the invariance of a normal varimax solution for more than two 


factors. The oblique normal varimax criterion is stated. A computational out- 
line for the orthogonal normal varimax is appended. 


In factor analysis, an analytic criterion for rotation is defined as one 
that imposes mathematical conditions beyond the fundamental factor 
theorem, such that a factor matrix is uniquely determined. Historically, 
the first such criterion was Thurstone’s treatment of the principal axes 
problem [10]: from any arbitrary factor matrix he suggested rotating under 
the criterion that each factor successively accounts for the maximum variance. 
But principal axes have seldom been accepted as psychologically very interest- 
ing ((9], p. 1389). The rotation problem for psychologically meaningful factors 
is usually handled judgmentally. Scientifically, however, this procedure is 
not very satisfactory: the ad hoc quality of subjective rotation makes 
uniquely determined factors impossible; only factors that are subject to the 
uncertainties and controversies besetting any a posteriori reasoning can be 
defined. In contrast, an analytic criterion for rotation would allow factor 
analysis to become a straightforward methodology stripped of its sub- 
jectivity and a proper tool for scientific inquiry. 


The Quartimax Criterion 


The first analytic criterion for determining psychologically interpretable 
factors was presented in 1953 by Carroll [1]. In an attempt to provide a 


*Part of the material in this a is from the writer’s Ph.D. thesis. I am indebted 
to my committee, Professors F. T. Tyler, R. C. Tryon, and H. D. Carter, chairman, for 
many helpful suggestions and criticisms. Dr. John Caffrey suggested the name varimaz, 
and wrote the original IBM 602A computer program for this criterion. 

I am also indebted to the staff of the University of California Computer Center for 
help in programming the procedures described in the paper for their IBM 701 electronic 
computer. Since their installation is partially supported by a grant from the National 
Science Foundation, the assistance of this agency is acknowledged. 
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mathematical explication of Thurstone’s simple structure, he suggested that 
for a given factor matrix, 


(1) f= : >95.05 
a<t 7 
should be a minimum, where j = 1, 2, --- , n are tests, s,¢ = 1,2, ---,r 


are factors, and a;, is the factor loading of the jth test on the sth factor. 
It appears that Carroll was motivated in writing (1) primarily by a close 
inspection of Thurstone’s five formal rules for simple structure ({12], p. 335), 
particularly the requirement that a large loading for one factor be opposite 
a small loading for another factor. 

In his original paper, Carroll provided two numerical examples of the 
application of his method. Without the restriction of orthogonality, these 
illustrations gave somewhat equivocal results—while the application of (1) 
appears to bring one close to the desired simple structure, the criterion has 
an obvious bias in being too strongly influenced by factorially complex tests. 

In the light of later developments, Carroll’s criterion should probably 
be relegated to the limbo of “near misses”; however, this does not detract 
from the fact that it was the first attempt to break away from an inflexible 
devotion to Thurstone’s ambiguous, arbitrary, and mathematically un- 
manageable qualitative rules for his intuitively compelling notion of simple 
structure. 

Almost simultaneously with Carroll’s development, Neuhaus and 
Wrigley [7], Saunders [8], and Ferguson [2] proposed what is usually called 
the quartimax method for orthogonal simple structure. Neuhaus and Wrigley 
suggest that a most easily interpretable factor matrix, in the simple structure 
sense, may be found when the variance of all nr squared loadings of the 
factor matrix is a maximum, i.e., 


(2) q: = [nr 2 > (aj.)? - (2; >. a3,)?|/n’r? = maximum. 


Saunders’ approach requires that the kurtosis (fourth moment over second 
moment squared) of all loadings and their reflections be a maximum, 


(3) Q2 = nr Zz Dd a}./( 20 > a?,)? = maximum. 


While Ferguson, basing his rationale on certain parallels with information 
theory, calls simply for 


(4) a = >> da}, = maximum. 


All these investigators are concerned with attaining a factor matrix 
with a maximum tendency to have both small and large loadings. While 
less obviously related to Thurstone’s rules than Carroll’s criterion, the 
emphasis on small loadings coincides with Thurstone’s requirements of 
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zero loadings. For orthogonal factors, criteria (2), (3), and (4) are equivalent 
because of the invariance of the sum of the communalities, >>;>.. aj. . 
(This term, as well as other constants, disappear when differentiated in the 
calculus problem involved in finding the required critical point.) 

Indeed, it turns out that they are also equivalent to Carroll’s criterion 
in the orthogonal case. Minimizing (1) is equivalent to maximizing (4) 
since the squared communality of a test is 


constant = (>.a3,)” = doa}, + 203.031, 


s<t 


and the sum of squared communalities over all tests is 


a Da, + 20 Daj.a5, 


7 s<t 


I 


constant 


= g; + 2f. 


Thus, since the quartimax criterion plus twice Carroll’s criterion is a con- 
stant, maximizing q3 is equivalent to minimizing f. 

Neuhaus and Wrigley realized that none of these criteria can be real- 
istically applied without the aid of an electronic computer—the calculations 
involved are too extensive for a desk calculator or punched card mechanical 
computers. Consequently, they programmed the quartimax method for the 
I]liac* and provided a rather extensive numerical investigation of the empirical 
properties of the quartimax method. 

Their results were perhaps more encouraging than Carroll’s. Under 
the restriction of orthogonality Carroll’s criterion (or the equivalent quartimax 
method) does not show nearly so obvious a bias as does Carroll’s criterion 
when the restriction of orthogonality is removed. However, as an explication 
of orthogonal simple structure, the quartimax method does have a systematic 
bias which will be more fully examined in the next section. 


The Varimax Criterion 


From the outset, the above methods consider all nr loadings simul- 
taneously. In every case, however, these criteria may be applied separately 
to each row of the factor matrix and summed over rows for the final criterion 
because of the invariance of the communalities. For example, Neuhaus and 
Wrigley could have defined the simplicity, say, of the factorial composition 
of the jth test as the variance of the squared loadings for this test, 


(5) gi=ir > (aj)? — (Lo a;,)"|/r’. 


*The Illiac is the University of Illinois electronic computer. Subsequently, the quar ti- 
max criterion has been programmed for the CRC-102A (Neuhaus), and the IBM 701 
(Kaiser). The varimax criterion described in the next two sections has been programm ed 
for SWAC at UCLA (Comrey), the IBM 701 (Kaiser), Illiac (Dickman), and the IBM 
650 (Vandenberg). 
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To obtain the total criterion for the entire factor matrix, (5) could then be 
summed over all tests to give 


(6) v= Lil > (aj,)° — (ho aj.) J/r*}. 


Maximizing q* is equivalent to maximizing g; , again because constant terms 
vanish when differentiated. 

Equation (6) perhaps provides some insight into the quartimax 
criterion—its aim is to simplify the description of each row, or test, of the 
factor matrix. It is unconcerned with simplifying the columns, or factors, of 
the factor matrix (probably the most fundamental of all requirements for 
simple structure). The implication of this is that the quartimax criterion 
will often give a general factor. Under requirement (5) there is no reason 
why a large loading for each test may not occur on the same factor. In practice, 
this tendency for the quartimax criterion to yield a general factor is most 
pronounced when the unrotated factor matrix has a pronounced general 
factor. 

From the simple structure viewpoint, an immediate modification of the 
quartimax criterion is apparent. Let us define the simplicity of a factor as 
the variance of its squared loadings, 


(7) vt = [n DG.) — (20 aj)" |r’. 


And for the criterion for all factors, define the maximum simplicity of a 
factor matrix as the maximization of 


(8) = Devt = Di fin 2D ai)* — (Le ai)"In’}, 


the variance of squared loadings by columns rather than by rows. 

Since a factor is a vector of correlation coefficients, the most interpret- 
able factor is one based upon correlation coefficients which are maximally 
interpretable. Those correlations which satisfy this condition are patently 
obvious: correlations of + 1, which indicate a functional relationship, and 
correlations of zero, which indicate no linear relationship. On the other hand, 
middle-sized correlations are the most difficult to understand. Thus, it is 
seen why v* in (7) could be maximized for the maximum interpretability or 
simplicity of a factor, and more generally, why the interpretability of an 
entire factor matrix could be considered best when (8) is a maximum. 

Criterion (8) is the original raw varimax criterion [4]. In the original 
proposal of this criterion, it was shown to be mathematically equivalent, 
in the orthogonal case, to minimizing 


(9) =D tin Dah — (Lai) (Lary, 


i.e., minimizing the covariance of pairs of columns of squared loadings and 
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summing over all possible pairs of columns for the criterion. Criterion (9) 
then bears the analogous relationship to Carroll’s criterion (1) that the 
varimax criterion (8) does to the quartimax criterion (6). 

Some distinctions between quartimax and varimax orthogonal solutions 
can be illustrated numerically. In Table 1 solutions for Thurstone’s eleven- 
variable box problem ([12], pp. 373-375) are given. It will be noted that the 
quartimax solution [7] could hardly be called a simple structure. There is 
a large general factor, and the second factor seems only vaguely concerned 


Table 1 


Thurstone's 1l-Variable Cox Froblem® 








Subjective Quartimax Raw Varimax 
(oblique) 

Test X 4 z X ¥ Z X Y Z 
x 909 05 00 68 65 05 GL  9Y" 36 
y oh 88 ol 83 -47 00 OS: 90) sn 
2 Gs... (05. “79 h2  -08 #879 1 27 88 
xy 62 63 =06 99 ll -Ob 6h. Fi eer 
z -05 5h 57 71 ~=—=LO 56 02 65 75 
«ey 82 . 37 <0 oo Wh 6 i ee ae 
xy? 35 76 02 96 «18 03 37 86 —t—«< 8’ 
2x + dy 53 71 -09 100 00 -07 ch 82 18 
(x2 + y2)2 52 71 -08 99 <-01 -07 53 8. 8618 
(x2 + 22)? S207 65 59 3868 60 09 77 
xyz 42 43 43 88 oh L5 48 58 65 





Decimal points omitted. 


with dimensions of boxes. On the other hand, the raw varimax solution 
closely parallels Thurstone’s original subjective solution, given the restric- 
tion of orthogonality. 

In Table 2 are solutions for Holzinger and Harman’s 24 psychological 
tests ([3], pp. 229-233). Both the quartimax [7] and the raw varimax methods 
seem to duplicate the subjectively rotated simple structure patterns. But 
the respective variance contributions of the factors are perhaps more interest- 
ing. It is seen that the dispersion of the >_; a3, for the subjective solution 
is less than the corresponding figures for the two analytic methods. In other 
words, Holzinger and Harman have made the factors a little more level or 
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Table 2 


Holzinger and Harman's Twenty-four Psychological Tests? 























Subjective Quartimax Raw Varimax Normal Varimax 

Test A B C D A B C D A B Cc D A B c D 
1 20. 32 62 20 S| 19° OO UOT “20 5 33 1; 19 «(67 Ge 
2 Oy 25°. 33 2, @7. 35. .ab a7 OF he 8 a0. O07 13-38 
3 wD. -a2-.53. a3 1 Gt oe we, Se 2S 15 02 Sk @& 
4 ie i Bie 4 36 O7 «(6 Ol 27 O08 S52 Ob 20 0 Sh @ 
5 7 25 2. 35 81 14 -02 -0h 76: 22° “22 6 7 2 22:98 
6 72 OS 28 25 81 03 0 7. a0 23 2b 76 10 2a 
7 oc) Re cies aime © | 8 07 -Oh -10 6; 45 20 “60 82 16 21 
8 Ss 26 36 ah 66 20 20 -Oh 60 25 31 05 Sh 26 38° a9 
9 76 -0h 29 30 86 -06 -02 10 2 Gi a2 39 60 G1. 2279 
10 = 666 69. 2b 23. 70 #12 lb 17 71 #+-08 #217 15 70 -06 2 
nu 27 61 -Oh 29 a1 6626 OL COB 22 6) 0 29 17 & o8 % 
12 BrP 8 16 69 19 «01 06 70 23 Ob 02 69 2am 
13 a 62 Ri 02 35 57 32 =08 2h 69. 39. =01 18 59 hl @ 
1h 23 19 =-02 418 32 19 -03 h2 26 20 O11 4h6 22 16 Oke oe 
15 11 #1 08 50 25 12 #10 4&5 57) aa 8 as 12. 07 2 
16 OS 22 gh .6 2p 23. Br. 37 7 oS: a 08 10 hi =e 
7 15 2h <3 62 206, Oe CS 20 23. 5S. i, 18 06 “ae 
18 a. 37 20 52 Pe Se HT oS 32. 32 Sk OO: -.2 32 ok 
19 i fe | ee 23 318 2 32 19 #36 22 -36 13 5S 2) a 
20 a. 26 6 UB S2 09 35 ik ue 42. 3. 35 L toe 
21 27 6 633) oh 35 3 3 lh 23 bo 40 20 15 8 2 2 
22 31 #12 «+440 210 53 Ob 30 2 bh 06 37 32 36 OL kl 3% 
23 31 29 Sh 25 55 19 kh o9 43 22 S52 . 26 5 2 Si 
2h 39 46 1 31 49 4310-20 LO 16 38 27 3h bh 22° @ 
P| Kee 343 292 268 236 559 2h2 196 1h2 431 260 26h 186 350 2hh 308 236 








Decimal points omitted. 


even in their contribution to variance than the analytic criteria. Of the 
two analytic criteria, the raw varimax solution has given a solution which is 
closer in this respect to Holzinger and Harman’s. It is also noteworthy that 
as a result of these differences the large loadings of the factors with the larger 
variance contributions for the analytic methods are larger than the large 
loadings for the smaller factors, and similarly, the small loadings for the 
larger factors are larger than the small loadings for the smaller factors. 
Holzinger and Harman’s subjective solution does not show this systematic 
bias; their solution gives a more equitable patterning of factor loadings. 
How this bias may be removed is indicated in the next section. This 
leads to a revision of the varimax criterion, which appears to have more 
important characteristics than merely satisfying the rules of simple structure. 


Factorial Invariance: Normal Varimax 


It seems reasonable to attribute the systematic bias seen in both the 
quartimax and varimax solutions of the Holzinger-Harman data and other 
examples [4] to the divergent weights which implicitly are attached to the 
tests by their communalities. When one deals with fourth-power functions 
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of factor loadings, a test with communality 0.6, for example, would tend to 
influence the rotations four times as much as a test whose communality was 
0.3. Thus, while the most obvious weights have been applied to the tests, 
namely the square roots of their communalities, after the fact it seems that 
there is probably a better set of weights—weights which would tend to 
equalize to a greater extent the relative influence of each test during rotation. 

There seems no rational basis for choosing among different weighting 
schemes. Let us then make the agnostic confession of ignorance which pervades 
any form of correlational analysis. For the purposes of rotation, weight 
the tests equally, in the sense that the lengths of the common parts of the 
test vectors have equal length. (The author is indebted to Dr. D. R. Saunders 
for this suggestion.) The varimax criterion could then be rewritten as 


(10) ds x {[n > (a;,/h3)” — [X (a;./hi)}\/n’}, 


where hj is the communality of the jth test. In contrast to (7) and (8), where 
the variance of the squared correlations of the tests with a factor is maximized, 
the variance of the squared correlations of the common parts of the tests 
(the reflections of the tests onto the common-factor space) with a factor is 
now being maximized. [Note from (10) that we are not advocating a permanent 
weighting of the tests by a weight inversely as the square root of their com- 
munalities. During rotation this weighting extends the common part of 
each test vector to unit length, but after rotation each of these vectors is 
shortened to its proper length by reweighting directly as the square root of 
the test’s communality.] 

As will be seen in Table 2, under this modification the varimax criterion 
(the normal varimax, since rotation is with respect to normalized common 
parts of tests) has effectively removed the small but disturbing bias in the 
raw varimax solution of Holzinger and Harman’s example. It also has been 
shown in a number of other examples [6] that the normal varimax does not 
seem to deviate systematically from what may be considered the best 
orthogonal simple structure.* 

Thus far, however, merely a numerical-intuitive basis for a weighting 
procedure which leads to “prettier’’ results has been provided. Such a basis 
is quite unsatisfactory theoretically. Indeed, this sort of ad hoe thinking 
could conceivably lead to a different set of judgmentally determined weights 
for any particular example—a situation as scientifically reprehensible as 
the subjective graphical methods. 

There is a more fundamental rationale for attempting to establish the 
normal varimax criterion (10) as a mathematical definition for the rotation 

*Professor Andrew Comrey has apparently reached the same conclusion in an exten- 
iy application of the normal varimax criterion to interitem correlation matrices of the 


MPI (personal communication). A further example, available from the writer, is the 
normal varimax solution of Thurstone’s classic PMA study[11] (dittoed). 
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problem. Consider the situation illustrated in Fig. 1. There are two clusters 
of tests, each of which is pure in the sense that the reflections of the test 
vectors of the cluster onto the two-dimensional, common-factor space are 
collinear. (While these clusters are drawn less than 90° apart, the following 
argument is perfectly general.) 


Ir 


es, 











Figure 1 


Case for which a normal varimax solution is invariant under changes in the composition 
of the test battery. 


It is shown below that the angle of rotation in a plane which maximizes 


(10) is 
Zin Duo; — Dw Levi) 
(11) res saeeenel™ p (u; — v;) — (0 w)° - (L0)] 


where 





U; => (a;,/h;)’ = (a;2/h;)’, 
and 
v; = 2(a;;/h;)(aj2/h;). 


Let n4 (n4 > 1) be the number of tests in the first cluster and ng (ng > 1) 
be the number of tests in the second cluster (n = n4 + Mg). It is readily 
apparent that all tests of the first cluster have the same values for u; and 9; . 
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Let these values be uw, and v, . Similarly let the values for the second cluster 
be wz and vz . In this case (11) reduces to 


Qn Np(Usva + Use — Use — Upva) 
2 2 2 2 . 
NaNg(Ua + Up — VA — VB — BWgtly + Ways) 





(12) @ = } arctan 


A most important result is shown in (12). The nang term may be 
cancelled, indicating that the angle of rotation does not depend on the 
number of tests in each cluster, i.e., for the case illustrated in Fig. 1, the normal 
varimax solution is invariant under changes in the composition of the test battery. 

This invariance property would seem to be of greater significance than 
the numerical tendencies of the normal varimax solution to define math- 
ematically the doctrine of simple structure. Although factor analysis seems 
to have many purposes, fundamentally it is addressed to the following 
problem. Given an (infinite) domain of psychological content, infer the 
internal structure of this domain on the basis of a sample of n tests drawn 
from the domain. The possibility of success in such inferences is obviously 
dependent upon the extent which a factor derived from a particular battery 
or sample of tests approximates the corresponding unobservable factor in 
the infinite domain. If a factor is invariant under changing samples of tests, 
i.e., shows factorial invariance ([12], pp. 360-361), there is evidence that 
inferences regarding domain factors are correct. 

The normal varimax solution, according to the above result, allows such 
inferences; regardless of the sampling of tests, for the problem shown in Fig. 
1 it is possible to infer precisely the domain normal varimax factors. This is 
not true for either the quartimax or raw varimax solutions “ince the angle of 
rotation is a function of n4 and nz, . 

Note that domain normal varimax factors are not said to be more 
meaningful than domain factors according to some different criterion; it is 
suggested that observed normal varimax factors will have a greater likelihood 
of portraying the corresponding domain factors. 

Although one often gets the impression that simple structure is the 
ultimate criterion of a rotational procedure, it is suggested here that the 
ultimate criterion is factorial invariance. The normal varimax solution was 
originally devised solely for the purpose of satisfying the simple structure 
criteria. But the fact that it shows mathematically this sort of invariance 
suggests that Thurstone’s reasoning was basically directed toward factorial 
invariance. The principle of simple structure may probably be considered 
incidental to the more fundamental concept of factorial invariance. This 
viewpoint renders meaningless the arguments concerning ‘psychological 
reality” of general factors, bipolar factors, simple structure factors, etc. 

Admittedly, the result (12) is for a special case. The correlations among 
the variables within each of the two pure clusters must form a_ perfect 
Spearman matrix, and the reduced correlation matrix as a whole must be 
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of rank two. Normal varimax does however give invariant results for two 
such Spearman clusters simultaneously, and consequently the normal varimax 
criterion is a two-dimensional generalization of the classic Spearman case. 
Obviously, the next step would be a generalization along the same lines to 
the r-dimensional case; thus far, however, work on this problem suggests 
virtually insuperable mathematical difficulties. 

To investigate numerically the tendency of the criterion to give factorially 
invariant solutions for r larger than two, again consider Holzinger and 
Harman’s empirical data. Taking their centroid loadings, the first five tests 
were rotated then the first six, etc., systematically until the analysis of 
all 24 tests was reached, as in Table 2. The results of this application of the 
normal varimax criterion are given in Table 3. There, the normal varimax 
loadings for the four factors as a function of the changing number of tests 
are given. 

Note that factors A and C are essentially invariant from the outset; 
the loading changes, while somewhat systematic, are negligible—24 appears 
to be essentially infinite. On the other hand, factors B and D show more 
movement before they become stable. The reason for this is readily apparent. 
For both B and D, this movement occurs only while they are underdetermined, 
i.e., only while they contain no appreciable loadings. However, once they 
pick up a test or two with high loadings, their ambiguous definition abruptly 
stops, and they settle down to exhibit a degree of invariance similar to the 


Table 3A 


a 
Normal Varimax Loading Changes for Holzinger and Harman's Factor A (n= 5, 6, e+, 2h) 








is 





Test oe od ® 20 22 32 25 GU a5 16 417. 28 89 20 2 2 eee 
1 28 30 «38 «4226 26 326 BSB 226 «6885 «(6 «6S. 26 96. 35, 15 235 | ee 

3 2 2 ° 23 202 2? BD Be PRB Ri Rm DB RB BR BD RD Bae 
3 aS 36 2S) 96. 6 «37 a7 8? A? 88 8 2g 0 6 26 86 ee 
4 A naerenaaa-wa a2 SH a° a 4a a hk Oe eee 
5 Dn BRD BK BE KBeHKHEKE EE BH HE HH SES Hae 
6 Bo?” 7 2 TF % FT Wt % 3 ct. 3: 3 % 5 sd S& @& 
7 63 8 83 82 -62 62 62 62 682 62 62 63 62° 62 82 °82: 825% 
8 59 58 55 «55 55 55 (655 CUS CSCS CSCS CSCS Sh Oh 
9 82 83 82 83 83 81 80 80 80 80 80 80 80 80 8 
10 28 35 37 27 26.26. 26 -25- 26 26.36 16. 35) Sore 
1 19 271 2 19. 248 38 #3 326 38 #18 36 «#326- 302 
12 03 62 ©3 03 03 (03 03:03. 03. 03. 03: G28 
13 is 29 39 #30 2 20 2.19 19 15 253mm 
14 33. 3h 22 Oh 2 ee 2 28 Ree 
15 12 420. YL. 32. 2? 02 22. 22: 2a 
16 09 09 10 09 09 09 09 09 6 
17 123 ah) @ wh WwW Ww Bees 
18 01 0 00 00 © Om 
19 bp UD Db Db Woe 
20 35. 35 35 35m 
21 16 16 #34638 
22 36 z a 
ob 3h 
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Table 3B 


Normal Varimax Loading Changes for Holzinger and Harman's Factor 5 (n = 5, 6, eeey 21)" 





n 





Oh Oh Ob 
20 21 
Lb 





2Decimal points omitted. 


Table 3C 


Normal Varimax Loading Changes for Holzinger and tiarman's Factor C (n = 5, 6, eeey 24)? 





Is 
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42 42h 
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Table 3D 
Normal Varimax Loading Changes for Holzinger and Harman's Factor D (n= 5, 65 eee, 24)~ 
n 
Test 5 6 7 8 ) 20-2 a? 33 Bh 5 RG: 34 8 20s 22 22. 23. 
1 ol -00 O a 67 2. Ob Mm. 02. 86. 23 9h. 35. 27 39-10: 38 1 lt ae 
2 ol Ol 02 @ 00 02 @ 08 Gf DD BB OF. OF 10  22..20 26 200 
3 “0 O01 O01 <00 O05 -00 O1 -00 -01 02 6.. OF O07. GB. 09 G9 03 0 08 08 
4 -03 -03 -03 -0h -01 -0h -03 -0h -0h O1 05 06 O06 7 O09 8 08 08 07:48 
5 “00 -06 =-03 <02 -09 -06 -06 -05 -05 O7 12 12 Yh wk 15 wb Ww 2 lu Y 
6 06 O09 O09 OS O05 O06 06 O5 dR A Oe ee. ey eee ee 2 22 ae 
7 “06 -06 -06 -10 -10 -09 -10 03 O7 O7 O9 O8 O09 O9 OF 9 08 08 
8 0, -1h -07 ~8 -07 -07 Ob 10 10 12 12 Ww WY 1 13 «#12 3 
9 15 11 12 12 6 22. 6 686C SCT aS 26°: 26 © 26 2 25 5 
10 00. 67 =<=00 02. 25.29 29. 23 2h 25 4, emer 2 2h (hy 
n OF: 22 RS.” Si PSL BR, Se! BR torr PSP as at oe 3 
12 1). <1 «00 0 06 o9 1 23 #22 12 ll lu &@ 
13 “17 -05 oO Ol Oh 07 08 08 07 97 07 «06 
1h TD: SD)... (0s. SB) 90) G9 -- «50 0 50 9 
15 Do wm.” Bf Se Oe 0. CS 50 50 
16 he b2 bh LS bh Lh bk Ok 
17 6h 6h 6h 6h ck Gh & 
18 su 55 55 Sk SSS 
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Decimal points omitted. 


other two factors, which had high loadings from the beginning. For n = 24, 
there appear to be good approximations to the domain normal varimax 


factors. 


The Oblique Case 


If the restriction of orthogonality is relaxed, it is impossible to apply 
directly the quartimax criterion (4) or the normal varimax criterion (10). 
This is because interfactor relationships are not considered when the criteria 
are in this form, and when applied all factors will collapse into the same 
factor—that one factor which best meets the criterion. However, Carroll’s 
version of the quartimax criterion explicitly considers interfactor relation- 
ships and an oblique solution is attainable. As suggested by (9), if 
(13) c= > {In Do (aj./hi)(ai./h) — (20 aj./hi)( 0 a3./hi)\/n"}, 
it may be shown that in the orthogonal case v = — 2c. This alternative form 
of the normal varimax may then be used to obtain oblique factors. The 
mathematical problem of minimizing (13) is exactly analogous to Kaiser’s 
[5] treatment for Carroll’s criterion. Computationally, the (iterative) solution 
involves finding the latent vector associated with the smallest latent root of 
a constantly changing symmetric matrix of order r. 
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Computational Appendix 


To compute an orthogonal normal varimax solution, the following 
procedure is suggested. The first step is to normalize the rows of the arbitrary 
reference factor matrix (e.g., principal axes or centroids) by dividing each 
element by h; . Rotation to the direction of the normal varimax factors may 
then be carried out with respect to these normalized loadings. 

The criterion (10) will be applied to two factors at a time. For this purpose, 
the following notation for an orthogonal rotation is convenient. 


ay i | Pe —sin | m ry. Y, | 
Z2 Yo| Lsin @ cos X; Y; 
Lvn Yn_! a Y,._ 














where x; and y;, the present normalized loadings, are constants, and X; 
and Y, , the desired normalized loadings, are functions of ¢, the angle of 
rotation. 

It is immediately seen that 


(14) X; = x; cos¢ + y; sin ¢, 
(15) Y; = —2; sind + y; cos ¢. 
Thus, 

(16) dX ;/de = Y;, 

(17) dY;/do = —X;. 


According to (10), in this plane, 
(18) on? yy =m D(X) — (XP +n DV) - (LY? 
should be a maximum. Differentiating (18) with respect to ¢, using (16) and 
(17), and setting the derivative equal to zero, 
(19) Pi Seo ) gilt gp RD OO 6 ae 6 gD a eee 
To solve (19) for ¢ in terms of x; and y; , substitute the values of X; and 


Y, from (14) and (15), consult a table of trigonometric identities, and, after 
a good deal of algebraic manipulation, 


(20) @ = } arctan 
Zin D(a’ — yay) — 2 (a? = y') 2 (2ey)] 
n{ ol’ — yy — Qey)’}} — (1 @ - YP - (2D ew} 
If u; = a} — yj and v,; = 2x,;y; , (20) reduces to the form (11) above. 


Of course, (11) or (20) is only a necessary condition for a maximum. 
By taking the second derivative of (18) sufficient conditions for a maximum 











200 PSYCHOMETRIKA 


may be found. These are summarized below. 


sign of numerator 








re i 
+ 0° to 0° to 
; +223° — 224° 
sign of 
denominator 
_| +223° | —223° 
to +45° to —45° 














The sign of numerator and denominator refer to the right-hand member 
of (20); the values in the cells refer to ¢. 

These single-plane rotations are made on factors 1 with 2, 1 with 3, --- , 1 
with r, 2 with 3, --- , 2 withr, --- , (r — 1) withr, 1 with 2, --- iteratively 
until r(r — 1)/2 successive rotations of ¢ = 0 are obtained, i.e., until the 
process converges. (It was shown [6] that v in (10) cannot be greater than 
(r — 1)/r, and since each successive application of (20) can result only in 
a non-decrease of v, this iterative procedure must converge.) After con- 
vergence, each normalized test vector is restored to its proper length by 
multiplying by h; . 

Since this article was accepted for publication, the author has prepared 
a detailed outline for coding an electronic computer program for the varimax 
criterion. This (dittoed) paper is available from the writer. 
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The specification of sample size is an important aspect of the planning 
of every experiment. When the investigator intends to use the techniques of 
analysis of variance in the study of treatments effects, he should, in specify- 
ing sample size, take into consideration the power of the F tests which will 
be made. The charts presented in this paper make possible a simple and 
direct estimate of the sample size required for F tests of specified power. 


A primary consideration in the design of any experiment is the specifica- 
tion of the number of subjects to be selected from the various treatment 
populations. This number should be such that the important statistical 
tests will be reasonably sensitive in detecting false null hypotheses. Statistical 
theory provides the basis for designing such tests; in many psychological 
and educational experiments sufficient preliminary information is available 
to permit an application of this theory. The purpose of this paper is to provide 
power function charts which will simplify the application of the theory and 
thus facilitate the specification of sample size in experiments employing the 
techniques of analysis of variance. 

The power of the statistical test in any experimental setup—that is, 
the probability of rejecting the null hypothesis when it is false—depends 
on the level of significance a at which the test is made, the number of obser- 
vations or subjects n on which data are available, and the degree of falsity ¢’ 
of the hypothesis under test. The latter factor is defined as the square root 
of the ratio of the variance of the treatment population means to the variance 
for error within the treatment populations. Symbolically, 


ard i= (us — wu) /ke 
o 








For every F test at a given level of significance in any given design, the 

power P against any specified alternative to the null hypothesis is uniquely 

determined by the value of n. Conversely, for every test there exists a value 

of n which will result in a test of specified power against a specified alternative. 
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In those experiments in which the power requirements of the F test can be 
rationally fixed against a specific alternative, it is possible to determine the 
appropriate sample size. It is for such situations that the present charts 
are intended. 


Nature of the Charts 


The charts presented in this paper are for use with tests of the main 
effects of treatments in experiments involving two to five levels of the treat- 
ment variable. The charts are strictly valid only for the completely random- 
ized design; however, they may be applied with relatively little error to tests 
of treatments effects in randomized block designs and factorial designs 
employing a within-cells estimate of error variance. A chart presents two 
families of three curves each. The families pertain to the .05 and .01 levels 
of significance; the curves within families correspond to power values of 
5, .7 and .9. 

A separate chart is provided for each value of k, the number of levels 
of the treatment variable, f, = k — 1, from 2 through 5. The chart and 
family appropriate for a given experimental test is entered with the param- 
eter ¢’ along the abscissa. The value of n, the number of observations required 
per treatment for a test of specified power, is read directly from the ordinate 
of the chart. 


Historical Development 


The distribution of the F statistic under hypotheses alternative to the 
null hypothesis was first considered by Fisher [1] and Wishart [9], who 
derived expressions for the noncentral F distribution in the form of the 
correlation ratio. Later Tang [8] derived the same result from the distribution 
of the noncentral x’. Tang also presented extensive tables of the power 
function. These tables are entered with the parameter ¢, defined as 


— Ja Me uw) /k 








For fixed values of a, ¢, f; , and f, the probability of retaining a false null 
hypothesis may be determined. Unfortunately, the interval of tabulation 
for ¢ is .50, an interval which is not sufficiently fine for satisfactory inter- 
polation. 

Following Tang’s procedure, Lehmer [4] tabulated the values of ¢ for 
a = .05 and .01, P = .7 and .8 over a wide range of f, and f, . These tables 
are quite complete within the power range considered; however they cannot 
be conveniently used in the planning of experiments. From the tables the 
experimenter can tell only that a projected test will have a power less than .7, 
between .7 and .8, or greater than .8 against a specified alternative. A greater 
range of power values would make such tables considerably more useful. 
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Patnaik [6] made an extensive study of the power function of analysis 
of variance tests. By the method of moments, he derived an approximation 
to the noncentral F distribution based on the central F distribution. This 
approximation is computationally feasible but somewhat tedious, especially 
if a number of power estimates are required. Its primary limitation to psycho- 
logical experimenters is the labor involved in utilizing Pearson’s Tables of 
the Incomplete Beta Function in obtaining power values. This limitation is 
especially marked in the many instances which demand interpolation within 
these tables. 

Pearson and Hartley [7] presented families of power curves for various 
combinations of a, f,; , and f, , which make possible a direct estimate of the 
power of analysis of variance tests. These curves, like the tables of Tamg, 
are entered with the parameter ¢. For any given experimental setup, the 
power of the test may be read directly from the ordinate of the curve. These 
curves are well suited to the evaluation of the power of any given test. They 
cannot be easily employed in the inverse manner, however, to indicate the 
value of » which should be adopted in order to secure a test of specified 
power. For this purpose, the experimenter must adopt the relatively in- 
efficient approach of making repeated approximations until the value of n 
has been estimated with sufficient accuracy. 

Nicholson [5] and Hodges [3] have derived general formulas for the 
computation of the power of the analysis of variance test when f, is an even 
number. The formulas involve the evaluation of terms in a certain series, 
the number of terms being dependent on the number of degrees of freedom 
for error. The latter feature is a serious practical limitation, for when f, is 
greater than 20, as it often is in psychological experiments, the evaluation 
becomes too laborious to be of practical utility. 

Fox [2] developed charts which overcome some of the objections to 
earlier works and facilitate the determination of sample size. These charts 
were constructed from the tables of Tang and Lehmer and are essentially 
graphs of constant @ for varying values of f, and f, . By a method of suc- 
cessive approximations, the value of n may be determined for a fixed value of 
a and a fixed value of P against a specified alternative hypothesis. These 
charts are somewhat laborious to apply, however, because of the iterative 
nature of the approximation for n. Also, the charts do not extend below 
f, = 3. For psychological experimenters, who typically deal with fixed 
treatments effects, this limitation considerably restricts their usefulness. 

In theory, the problem of evaluating the power of the test of treatments 
effects in the simpler analyses of variance has been completely solved. Exact, 
approximate, and graphical solutions have been derived. However, neither 
the computational formulas nor the graphical solutions make possible a 
simple, direct, noniterative approximation of the sample size required for 
a test of specified power. The charts presented in this paper permit this 
direct approximation for n, and hence should be of considerable value. 
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Construction of the Charts 


Each chart in this series presents, for a = .05 and .01, a family of three 
curves which correspond to P = .5, .7, and .9. The numerical calculations 
for the coordinates of the points on the curve P = .7 were carried out from 
the tables of Lehmer; the calculations for the remaining curves were based 
on data read from the charts of Pearson and Hartley. The three basic steps 
in the calculations were as follows. 


(1) Determine (from chart or table) pairs of values for @ and f, for a 
specified value of P, f, , and a. 

(2) Solve f. for n from the relationship n = 1 + f./k, where k is the 
number of treatments and n the number of observations per 
treatment. 

(3) Divide ¢ by +~/n to obtain ¢’, 


The pairs of coordinates for n and ¢’ were then plotted and smooth curves 
fitted through the points. 


Example 


An experimenter wishes to compare the level of mastery reached by 
three groups of college subjects who mémorize a list of paired adjectives 
under three levels of motivation. From a tentative theoretical formulation of 
the learning task the experimenter predicts the following array of mean 
differences for the treatment populations at the three levels: 


M, — Mr — 5.0; 
M, sas Miy = 8.0; 
Mi — Mir = 3.0. 


Against this alternative to the null hypothesis the experimenter wishes a 
power of .90 for a test made at the 5 per cent level. Previous experimentation 
with this list has given rise to an error variance of 100.0, a value which may 
be taken as a population parameter for this purpose. 

From the array of differences the variance of the treatment population 
means can be computed equal to 10.89. The value of ¢’ is therefore equal to: 
.33. Entering Figure 2 with this value, the required number of subjects is 
equal to 40. 


Note on the Generality of the Charts 


The charts presented in this paper are strictly valid only for the test 
of main effects in the completely randomized design. However, values of n 
read from these charts underestimate only slightly the values of n which 
would be required in the randomized block or factorial designs. 
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The specificity of the charts stems from the unique relationship which 
holds between n and f, in each experimental design. For example, for the 
completely randomized design (and the one used in the construction of 
these charts) the relationship is 


1 + fo/k. 


n 


In the randomized block design 
i fo/(k fe, 1). 


For the test of the factor with k levels in the k X h factorial design (mean 
square within cells being used as the error term) the relationship is 


n= h+ fo/k. 


Because of the differences among these relationships, the numerical 
relationship of ¢ to ¢’ varies from one design to another, and charts based 
on that which holds for the completely randomized design will be only 
approximately correct for the other setups. However, for values of f, > 20, 
the relationship of ¢ to ¢’ is almost identical for all three designs. We may 
demonstrate the relatively small error involved in using the present charts 
for planning randomized block and factorial designs by applying the charts 
to two examples which tend to maximize the extent of the inaccuracy. This 
occurs in the randomized block design when k = 2; it occurs in the factorial 
design employing a within-cells error term and proportional frequencies 
when the number of measures per cell approaches 2. According to Figure 1, 
a completely randomized design involving two levels of the treatment variable 
and a = .05 will require n = 11 (f, = 20) for P = .90 against ¢’ = .725. 
The value of n which is actually needed in a randomized block design for 
P = .90 is approximately 12.0. The comparable value for a 2 X 6 factorial 
design is 11.9. 

This discrepancy, which for even this extreme case is probably of little 
consequence in the planning of most experiments, is considerably smaller 
for larger values of f, and f, . Therefore, for practical purposes of approxi- 
mating the necessary sample size in randomized block and factorial experi- 
ments, the tables presented in this paper would seem sufficiently precise. 


n 
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Three methods of rotation (the pe. the Thurstone analytical, 
and the direct-rotational) were applied to the matrix of centroid loadings 
for 35 variables, to determine which method is the most efficient from the- 
oretical and practical standpoints. The direct-rotational method provided 
the most information for determining the rank of the configuration and was 
most economical with respect to time required to reach a rotational solution. 
The analytical method required the least number of judgmental decisions 
and was the most objective. The graphical method was the most laborious 
but had a slight advantage with regard to the number of near-zero loadings 
in the rotational solution. 


Since 1932, the year in which L. L. Thurstone introduced rotational 
procedures to factor analysis, many different methods of rotation have 
been developed. A survey of the literature reporting factorial studies has 
revealed that graphical methods have been employed most frequently to 
accomplish the rotations even though these methods are relatively slow, 
laborious, and require numerous skilled judgments. Until the present time, 
however, they have been the most dependable methods. The majority of 
the recent proposals for the solution of the rotational problem have been 
analytical procedures with emphasis on objectivity. 

Thurstone [9] and Carroll [1] have developed analytical methods by 
which solutions may be obtained on orthogonal or oblique axes. Saunders 
{8] and Neuhaus and Wrigley [7] have developed variants of Carroll’s method 
which yield orthogonal factors. With the exception of Thurstone’s method, 
which requires judgment in selecting the initial weights for a factor, the 
procedures are sufficiently systematic so that electronic computer programs 
for them can be written. Differing in approach, a few methods have been 
advanced that are of particular theoretical interest because they have proposed 
to reduce the problem to a single rotation. Of these methods, the one offered 
by Harris [5], based upon geometric models, has shown considerable promise 
for future development. 

This paper reports an investigation which compared three oblique, 
rotational solutions to a battery of 35 aptitude, achievement, and back- 
ground variables [2]. The graphical, the Thurstone analytical, and the 
direct-rotational methods were applied. 
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The study attempted to assess the validity of the following propositions 
by comparing the results from the solutions derived by the three rotational 
methods. 

(1) The rank of the factor configuration is determined with greater 
accuracy by the direct-rotational method than by graphical or analytical 
methods. 

(2) The solution obtained by the direct-rotational method yields a 
result which is a closer approximation to simple structure than the results 
obtained by the graphical and analytical methods. 

(3) The solution by the direct-rotational method requires less time 
to reach an approximation to simple structure than do the graphical and 
analytical solutions. 

(4) The direct-rotational solution requires a smaller number of skilled 
judgments on the part of the investigator than do the graphical or analytical 
solutions. 


Procedure 


A comprehensive test battery was administered to 881 male, basic 
airmen at Lackland Air Force Base, San Antonio, Texas. The mean chrono- 
logical age of the airmen was 18 years; their mean educational attainment 
was the eleventh grade. 

Product moment correlations were obtained for 35 variables selected 
from the Army General Classification Tests, the Airmen Classification Test 
Battery, the Adjutant General Mechanical Aptitude Test, the Differential 
Aptitude Tests, the Gray-Votaw General Achievement Tests, the lowa High 
School Content Examination, the Otis Quick-Scoring Mental Ability Test, 
the Sims Score Card for Socio-Economic Status, and education in years. 

The matrix of intercorrelations was factored by the centroid method. 
Several empirical criteria, such as Tucker’s Phi, Coombs’ Criterion, and the 
range of the centroid loadings indicated the rank of the correlation matrix 
to be eight. (The intercorrelation and centroid factor matrices have been 
published in Fruchter [2].) 


The Graphical Solution 

The graphical method made use of the eight columns of the centroid 
factor matrix F, . Each column or axis of F, was plotted against every other 
column. Each plot was examined to determine whether a rotation of a pair 
of columns or axes was indicated. After a pair of axes was rotated, the graphs 
involving these axes were replotted. The new plots were again inspected 
to determine the need for additional rotations. The process was continued 
until the projections on the orthogonal axes approximated simple structure 
as closely as possible. Since several plots indicated an oblique configuration 
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of factors, and in order to make relative comparisons with the other solutions, 
an oblique solution was obtained graphically from the orthogonal solution. 
The oblique solution, in which the factors are identified as verbal, numerical, 
mechanical, perceptual, visualization, academic information, correct English 
usage, and socio-economic background, is shown in Table 1, and the cosines 
of the angular separations of the axes in Table 4. 

The apparatus designed by Zimmerman [10], consisting of a drawing 
board, T-square, and a triangle, was used in the rotational procedures. This 
eliminated the necessity of converting the projections to numerical values 
after each rotation and of replotting the numerical values. Eighty-two rota- 
tions of pairs of axes were made in obtaining the oblique solution by this 
method. 


The Analytical Solution 


Thurstone’s analytical procedure is a single-plane method since the 
coordinate hyperplanes are determined one at a time. The procedure was 
started by selecting and normalizing test vector 15 (Numerical Operations), 
which served: as a trial vector to find the first reference vector. Test 15, as 
required in the procedure, has some relatively high and low correlations in 
the correlation matrix. The trial vector was then adjusted to a position A, 
which is the normal that defines the first hyperplane. 

The test vector 31 (Gray-Votaw Reading), which has a low projection 
on the first reference vector A; , was selected to become the second trial vector. 
(A projection was considered low when its absolute value was low relative 
to the rest of the test projections on the same reference vector.) It was 
employed to determine reference vector A;; . The test vector 18 (Tool Func- 
tions) with low projections on A; and A;; was selected as the third trial 
vector to find reference vector Ayr; . 

This process was continued until the number of reference vectors so 
determined was equal to the number of factors in the centroid factor matrix. 
The eight factors identified by the analytical solution were the same as those 
identified in the graphical solution, although two of the factors, correct 
English usage and academic information, did not appear to be independent 
of the verbal factor according to the requirements for simple structure. 
The rotated factor matrix obtained by the analytical solution is given in 
Table 2 and the cosines of the angles between the axes in Table 5. 


The Direct-Rotational Solution 


The method outlined by Harris [4, 5] was used to obtain the third 
solution. In this method the assumption is made that a given factor con- 
figuration is consistent with a specific geometric model. The postulated 
figure is a tetrahedron for 3-factor problems while the hypertetrahedron is 
the analogous model for higher-dimensional problems. The objective is to 
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locate the primary axes at the intersections of the hyperplanes. The loadings 
of points at the intersections of the hyperplanes provide direction numbers 
which constitute a basis for deriving the transformation matrix. Once these 
points are established, a single rotation is required to obtain the rotated 
loadings of the variables on either primary or simple axes. 

The test vectors of the factor matrix F. were extended to the mean 
of the first centroid loadings rather than to unity, as is usually done, in 
an effort to reduce distortion introduced by extending vectors with low 
values on the first centroid. The extended loadings were plotted on combi- 
nations of pairs of axes. The plots were inspected to establish the locations 
of the eight primary axes. The same factors were identified in this solution 
as in the graphical and analytical solutions. However, the high intercorrel- 
ations of the verbal, academic interest, and correct English usage factors 
plus several extremely high values in the inverse of the intercorrelation 
matrix of the eight primary factors indicated that the axes were not suf- 
ficiently independent to represent separate factors. The possibility of a 
seven-factor solution in which the verbal and correct English usage factors 
of the previous solution were combined was investigated next. The two 
factors in the verbal domain, verbal and academic interest, were still too 
highly correlated in this solution to be considered separate dimensions. 

A solution based on the assumption that only six independent factors 
were present in the test battery was then tried and accepted. This solution, 
which is presented in Table 3, satisfies the criteria for determining the rank 
of a factor configuration. Briefly, the criteria stipulate that: (1) the ex- 
tended vector configuration should be consistent with the assumed geo- 
metric model; (2) the intercorrelations of the factors should be small enough 
to give evidence of their independence; and (3) the inverse of the inter- 
correlation matrix of the factors should be computed satisfactorily, ie., 
most of the entries should be small, preferably less than unity. The cosines 
of the angles of the six factors which were identified as verbal, numerical, 
mechanical, perceptual, visualization, and socio-economic background are 
given in Table 6. 


Results 


The rank of the facter configuration by the graphical method was 
judged to be eight. The sole criterion applied during rotation specified that, 
if one or more of the factors achieved only small loadings (i.e., between 
+ .20), it would be residualized and the rank would have been less than 
eight. The correct English usage factor was not identified with confidence, 
although it did not residualize. The application of simple structure criteria 
and an inspection of a set of plots of the rotated factors served as a final 
check on the adequacy of the rotational solution. 


| 


TABLE 
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TABLE 4 


Cosines of Angular Separations Between Simple Axes 
Graphical Solution 
(All entries’to three decimal places ) 














II III IV V VI VII VIII 
V N Me P Vz AI CU SE 
I 1000 
II -246 1000 
Ill 095 -040 999 
IV 092 -234 106 1000 
V -094 -138 -433 -143 1000 
VI -425 -056 299 -258 -104 1000 
VII 006 -085 009 051 -120 -196 1000 
VIII -160 -043 063 009 098 -101 -036 1000 
TABLE 5 


Cosines of Angular Separations Between Simple Axes 
Thurstone Analytical Method 


(All entries to three decimal places) 











t II III IV V VI VII VIII 
N V Me Vz SE P AI cu 
I 1000 
II -296 1000 
III 111 -025 1000 
Iv O46 -157 -236 1000 
Vv 080 -059 05 152 1000 
vI 166 024 104 -148 003 1000 
VII -153 819 194 “440 011 096 1000 


VIII -353 831 022 -195 “144 125 872 1000 











VIII 
SE 


) 


! 


vill 


1000 
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TABLE 6 
Cosines of Angular Separations Between Simple Axes 
Direct Rotational Method 


(All entries to three decimal places) 











II III IV V VI 
V N Me P Vz SE 
I 1000 
Il 494 1000 
III -033 -112 1000 
IV 086 -328 136 1000 
Vv -172 097 -467 -628 1000 
VI -520 278 -176 -263 113 1000 





The analytical solution identified the same eight factors. However, 
the application of the simple structure criteria placed the correct English 
usage and academic information factors in a doubtful category [3]. The 
similarity of the verbal, correct English usage, and academic information 
factors was indicated by a questionable fulfilment of the requirement that 
for every pair of factors there should be several variables with projections 
on one factor vector but not on the other. There was also a question as to 
whether the number of tests was small enough to conform to the requirement 
that for every pair of factors only a small number of variables should have 
appreciable loadings on any pair of factors. 

Three separate solutions, based on the assumptions that the rank of 
the factor configuration was eight, seven, and six, respectively, were obtained 
by the direct-rotational method. The hypothesis regarding six factors was 
accepted because the solution produced a figure consistent with the postulated 
geometric model; the intercorrelation matrix of the six factors gave evidence 
of their independence; and the inverse of the intercorrelation matrix of the 
six factors was computed satisfactorily. 

The disagreement in the rank of the factor configuration may be at- 
tributed to the restriction imposed by the direct-rotational method in regard 
to the location of reference axes. The criteria were strict and tended to 
determine conservatively a rank characterized by highly independent factors. 
On the other hand, the graphical method permitted more latitude in locating 
the reference axes of the verbal domain, so long as the simple structure 
criteria were not violated. The placement of reference axes was somewhat 
more objective by the analytical method. One reference axis or factor was 
located at a time, and the rank was determined, in the final analysis, by the 
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application of the simple structure criteria. The rank of the factor configura- 
tion seemed to be most precisely determined by the direct-rotational method 
since criteria for the locations and angular separations of the primary axes 
were more definitely specified by the additional requirements. 

The comparison of the six factors common to the three solutions indicated 
no appreciable differences in approximating simple structure. The largest 
number of vanishingly small loadings (a;; = -+.20) on each of the six factors 
was used as a discriminant. The graphical solution had a slight edge over 
the other two solutions in this respect. 

The solution obtained by the direct-rotational method was most eco- 
nomical with respect to amount of time. It required 88 man-hours to reach 
a solution which was approximately 30 and 50 hours less than the analytical 
and graphical solutions respectively. 

The analytical method was least demanding of judgmental decisions 
on the part of the rotater. Judgment was required in the selection of a variable 
to serve as the initial trial vector for each factor. The remainder of the method 
consists of routine statistical calculations. The major judgmental portion 
of the direct-rotational method consisted of locating the intersections of the 
hyperplanes. This portion was concentrated near the beginning of the pro- 
cedure, with the rest of the method consisting of routine statistical calcu- 
lations. The graphical method required a judgment for each rotation of a 
pair of axes and was most demanding in this respect. 


Conclusions 


The direct-rotational method, in batteries for which it is suitable (i.e., 
batteries with a central factor on which all the variables have at least moderate 
positive loadings), provides a better basis for determining the rank of the 
factor configuration and is more economical with respect to time required 
to reach a rotational solution than the other two methods. The analytical 
method is most useful where it is desired to hold judgmental decisions to a 
minimum and to turn the calculations over to a statistical clerk or computer. 
The graphical method is the most laborious but had a slight advantage in 
the present study with regard to the number of near-zero loadings in the 
rotational solution. 
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ANALYSIS OF VARIANCE FOR CORRELATED OBSERVATIONS 


RayMonp O. CoLtier, JR. 
UNIVERSITY OF MINNESOTA 
Two different linear models are presented for the four-dimensional 
classification —_— in which correlations exist between certain pairs of 
observations. Except for the assumption of correlated observations, classical 
assumptions associated with classification systems are made. The models 
considered are modifications of those which underlie the split-plot design 
and the split-split-plot design. In the first model the correlations between 
observations of the levels of one dimension are all set equal to p. In the 
second model the observations of the levels of one dimension are assumed 
correlated to degree pi, whereas the observations of a second dimension are 
correlated to degree p:. Analyses for the two models and tests of hypotheses 
for various parameters are indicated. 


In the social sciences it is generally agreed that the analysis of data 
involving several observations on a subject is of basic interest and importance. 
Typically, several different variables are observed for every experimental 
subject. A special case arises when several repeated or multiple observations 
are made on a single variable for every individual subject, each successive 
observation usually being associated with a different level or combination 
of dimensions of the design. In both kinds of settings it is usual to assume 
that the observations associated with a single subject are correlated. 

Concern over such experiments involving repeated measurements was 
manifested quite early in statistics and the social sciences. Recent con- 
sideration of this problem has been made by several writers. Kogan [9] 
outlined an analysis for a design in which several groups of subjects were 
administered different experimental treatments with each individual subject 
being measured on several trials. Edwards [4] devoted a chapter to the 
analysis of designs which involved repeated measurements on the same 
subjects. Later, in surveying the status of experimental design in psychology, 
Kogan [10] pointed out the relevance of the split-plot design in experi- 
ments for which repeated observations were taken on each subject. Perhaps 
the most comprehensive treatment of the design and analysis of experiments 
involving repeated measurements for two- and three-dimensional setups 
has been given by Lindquist [12] in his chapter on mixed designs. An early 
example of the application of the split-plot principle in an experiment on 
drawing involving repeated or correlated observations was given by Hoyt, 
Stunkard, et al. [7]. Moonan [13, 14], in considering test reliability and 
experimental design, presented a model which involved observations assumed 
to be correlated. 

Perhaps the first presentations of the analyses of designs in which 
correlations between observations could be assumed were made by Yates 
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[18, 19, 20]. In these papers Yates presented the analysis of the split-plot 
design as it applied to agricultural experimentation. Later, Nandi [15] 
supplied a transformation useful in the derivations of possible tests of hypo- 
theses in split-plot designs. The work of Halperin [5] parallels that of Nandi, 
the pattern of correlation coefficients assumed by both writers being similar. 
Notable among the many books treating the split-plot design and variations 
thereof are those by Cochran and Cox [2] and Kempthorne [8]. In addition 
to the above, a more comprehensive list of references pertaining to the split- 
plot design and the analysis of designs with correlated observations has been 
made by the author [3]. 

With few exceptions work relative to the split-plot design has been 
done by workers in fields other than the social sciences. The result has been 
that the ramifications of this design and certain variations of it are not 
greatly appreciated by workers in the social sciences. Moreover, the simple 
split-plot design as usually presented, may be generalized to higher di- 
mensional designs only if a good deal of analysis of variance insight and 
intuition is employed. Finally, it has been practically impossible to find 
the analyses of such generalized designs presented in the literature, let alone 
the underlying assumptions listed. 

It is the purpose of this paper to study a class of designs in which data 
of the sort mentioned above are involved. The analyses included will fall 
into the general setting of the analysis of variance as applied to experimental 
design. 


A Four-Dimensional System with Constant Correlations Between 
Levels of One Dimension 


It should be sufficient for many investigations to consider layouts in 
which there is a maximum of four dimensions and equal frequencies. in the 
subclasses. The different dimensions may refer to experimental treatments 
or simply to ways of classification. Suppose that limitations on the possible 
number of subjects available, constraints imposed by scheduling, or the 
desires or interests of the experimenter, require that multiple measurements 
be taken on each individual subject of a sample. Suppose further that each 
of these observations refers to an observation made under a level of a particular 
dimension of the design. If the dimension is experimental, each subject is 
customarily presented the treatments in random order so as to balance 
sequence effects. Under such circumstances it will be possible to make certain 
assumptions and arrive at various tests of hypotheses. 

Let X,,.:, Stand for the variable observed on the gth individual for 
the rth level of dimension r, the sth level of dimension s, the tth level of 
dimension ¢, and the uth level of dimension u. Let the dimension to which 
subscript wu refers be that which involves correlated measures. Each subject 
is observed under all of the levels of u, although under only one of the combi- 
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nations of r, s, and ¢. The subscripts on the random variables, X,,,., , will 
be simple to follow if g = 1, 2, --- ,Q;r = 1,2, ---,R;s =1,2,---,8; 
t= 1,2,---,7;andu = 1, 2, --- , U. Thus, the QRST subjects, i.e., Q 
different subjects in each of RST subclasses or dimension level combinations, 
are measured on each of U levels of one dimension. 

As previously stated assume that correlations exist between pairs of 
observations taken on the same subject. One model which may be utilized 
assumes that the random variables X,,.,, are QRSTU-variate normally 
distributed with variances and covariances as follows. 


(1) V[Xeretu] = 0°; 

COV [Xaretu » Xarrrert'u'] 
=poe, fr g=q', r=’, s=e’, t=, uxtw’, 
= 0 otherwise. 

Let the expected value of a single observation, given as a linear function of 


fixed constants, be identical to that usually expressed in Model I of the analysis 
of variance. 


(2) E(Xeraw) = w + a1, 7) + a(2, 8) + a(8, t) + a(4, u) + BCL, 18) 
+ (2, rt) + B(3, st) + B(4, ru) + (65, su) + B(6, tu) 
+ y(1, rst) + y(2, rsu) + 7(8, rtu) + y(4, stu) + d(rstu) 
where the fixed constants are defined as follows: 


uw refers to the general effect; 

a refers to the main effects of the different dimensions indicated; the 
numerals being employed to distinguish the’main effects of different 
dimensions when specific values of 7, s, ¢, or u are considered; 

8 refers to the first-order interactions between the two dimensions 
indicated ; 

y refers to the second-order interactions between the three dimensions 
indicated; and 

6 refers to the third-order interaction constants between all four 
dimensions. 


Since the fixed model is assumed, the conditions concerning the distribution 
of the X,,,:, are equivalent to 


X arate — E(X retu) + €grstu 9 


where the ¢,,,:, refer to random effects. The random components, €,,.:. , are 
assumed to be QRSTU-variate normally distributed with 


(3) E€eretu) = 0, 
Vere - a”, 
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and 
COV (Eorstn » €'r'a't's’) = po, ,forg=q,r=r,s=e,t=t,uxeu’ 
= 0 otherwise. 


In addition, the following parametric restrictions are imposed: 


Da= b= Vv=DVsi=0. 


Thus the sum of parameters over a parscript involved is zero. 

In effect, the assumptions made on the variances and correlations state 
that the variance of the random observation X,,.:, is o°, the correlations 
between any two observations made on the same subject are constantly p, 
but the correlations between two observations not made on the same subject 
are zero, i.e., these observations are stochastically independent. 

It is possible, by means of an orthogonal transformation, to transform 
the original variables X,,,,, to two sets of variates which are independent 
within a set and between sets and which have constant variances within a 
set, but possess variances which differ from set to set. This transform, due 
to Nandi [15], follows. 


Ww or u=1 ’ 
arst(w+1) ~ 


w+ Ww 
(4) (w=1,2,---,U- 1): 


oe 
Yorstu = VU Jairo 


From this transformation, in set (1) there are @RST(U — 1) variates 
Yoreru (& = 1, 2, --- , U — 1), which are normally and independently dis- 
tributed with constant variance 0? = o° (1 — p). If the Y,,.:y are labeled 
Zorstu , it follows that in set (2) there are @RST variates, Z,,.:y , normally 
independently distributed with constant variance o2 = o” [1 + (U — lp], 
which are also independent of variates in set (1). Since the transformation 
of (4) is orthogonal it also follows immediately that the sum of squares 
associated with the variates of set (1), the Y,,.:., (uv = 1, 2, --- , U — 1), is 


(5) =. ¥ » p> » ig _ ECT .iodT 
= DULL UU, — Ae = ald, a) — ald, rw) — 86, su) 


— B(6, tu) — y(2, rsu) — y(3, rtu) — y(4, stu) — d(rstu)]’, 


asiies a 





i >. Mes 
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and that the sum of squares associated with the variates of set (2), Zyrs:a , 18 


(6) > x > > [Zaretv a E(Zorev)) 
=U DEED [Ae — uw — all) — 02,9) — 08, 9 — acts) 


ae B(2, rt) be A3, st) res v(1, ra |, 


where, in the above expressions, X,-s:. = >.7-; Xqret. and any sum expressed 
without limits of summation is to be interpreted as a sum over the entire 
range of the subscript. 

Noting that the parameters in (5) are involved there only, and a similar 
statement is true of (6), normal regression theory, e.g., as presented by 
Wilks [17] or one of other equivalent techniques may be utilized to derive 
tests of hypotheses on parameters involved in the two variate sets. The 
quantities necessary to test these hypotheses are presented in the analysis 
of variance of Table 1 and the notation employed in the specification of the 
test statistics follows this table. 

Tests of hypotheses are carried out as is usual in the analysis of the 
split-plot design. That is, there is an error term for each of the two sets 
which is used to test hypotheses in a particular set. For example, the test of 
the hypothesis, H, : 6(rstu) = 0, in set (1) is made by forming a ratio of the 
mean square related to the hypothesis and the mean square due to error (1), 


8.8.[d(rstu) ]J/(R — 1)(S — 1)(7 — 1)(U — 1) 
S.8.(Error 1)/(Q — 1)(RST)(U — 1) 


where 8.8. [6(rstu)] and S.S. (Error 1) refer to sums of squares appearing 
in Table 1. F(6), under H, , is distributed as Snedecor’s F distribution with 
(R — 1) (S — 1) (T — 1) (U — 1) and (Q — 1) (RST) (U — 1) degrees of 
freedom. Therefore, to test Hy above, with a significance level of a, require, 
as is usual, that F[é] exceed the upper a-point of this particular F distribution. 

Tests of other hypotheses in set (1) are made by forming analogous 
F ratios using the denominator as in (7) above (see [8] pp. 91, 375). Concerning 
the pooling problem the reader should refer to a discussion of some of the 
consequences of pooling nonsignificant interaction effects with error recently 
given by Binder [1]. It should also be noted that a testing procedure recently 
proposed by Hartley [6] seems promising here and elsewhere in this paper. 
Tests of hypotheses on parameters involved in set (2) follow perfectly analo- 
gous lines. Thus, as an example, the test of the hypothesis, H, : y (1, rst) = 0, 
is made by means of the F ratio, 


8.8. fy, rs J/(R — I(S — 17 — 1) 
8.8.(Error 2)/(Q — 1)(RST) 





(7) F(8) = 





(8) Ffy(1)] = 
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It is worth noting that the final result has been that of dividing the total 
sum of squares into two portions, namely the sum of squares associated 
with Y,,...,u = 1,2, ---, U — 1, given by 


(which is seen to be the sums of squares between observations on the same 
individual summed over all individuals) and the uncorrected sum of squares 


between individuals, 
Sete. 


TABLE 1 


Analysis of Variance for a Four Dimensional Layout with 
Constant Correlations Between the Observations 
of Levels of One Dimension 














Degrees 
nea of Sum of Squares 
Freedom 

Total 

Set (1) QRST(U-1) al-p! 
(Within 

individuals) 
a(4,u) U-1 O-p 
B(4,ru) (R-1) (U-1) i-L-o+p 
B(S,su) (S-1) (U-1) j-m-o+p 
B(6,tu) (T-1) (U-1) k-n-o+p 
7(2,rsu) (R-1) (S-1) (U-1) c-f-1- j+l+m+o-p 
¥(3,rtu) (R-1) (T-1) (U-1) d-g-1-k+f+n+o-p 
7(4,stu) (S-1) (T-1) (U-1) e-h- j-k+m+n+o-p 
8(rstu) (R-1) (S-1) (T-1) (U-1) a-b-c-d-e+f+g+h+i+ J+k-f-m-n-o+p 
Error (1) (Q-1) (RST) (U-1) a'-b!-a+b 
Total 

Set (2) QRST ” 
(Between 

individuals) 

u 1 p 

a(l,r) R-1 £-p 
a(2,s) S-1 m-p 
a(3,t) T-1 n-p 
B(1,rs) (R-1) (S-1) f-f£-m+p 
A(2,rt) (R-1) (T-1) g-L-n+p 
B(3,st) (S-1) (T-1) h-m-n+p 
7(T,rst) (R-1) (S-1) (T-1) b-f-g-h+l+m+n-p 
Error (2) (Q-1) (RST) b'-b 
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Similarly, the error sum of squares in set (1) may be observed to be the sum 
of the interactions of individual and dimension u, whereas the error sum of 
squares in set (2) is simply the sum of squares between individuals in a sub- 
class summed over subclasses. The sums of squares associated with the various 
effects are computed in the usual fashion. 

The parameters listed in the “Source of Variation” column of Table 1 
should be interpreted as referring to the source of variation associated with 
the test of hypothesis on the parameters of the effect being considered. Thus 
a(4, u) refers to the source of variation connected with Hy : a(4, u) = 0. 

The mean squares have not been given in Table 1 since they are computed 
in the usual fashion. The expected mean squares for error (1) and error (2) 
may be shown to be 


ao =o(1 — p) and of =o [1 +(U — lp], 


respectively. 
In Tables 1 and 2 the notation employed is given by 


(9) a= > > > > Xi a/Q; b= > > > X?,,.../QU, 
c= p> > x xt/QT; d= > > x X?,.14/QS, 
e= > dX > X?.41u/QR, j= > x X2::/QTU, 
g= > > X?,.../QSU, h= > Dy Xtu./QRU, 
i= p> > X?,..u/QST, j= > > X?.,../QRT, 
k= 3X Xtw/QRS, L= 2 X%,.../QSTU, 

m = > X2,../QRTU, n= x X?..,./QRSU, 
o= 2) X%.../QRST, p = X?..../QRSTU, 
e@ EEE EEL = EEE EXult, 


c’ > x Xo Xie./TU. 


A dot used here in place of a subscript refers to the fact that the sum has 


been taken over that subscript. 

For the comparison of individual means by use of one of the techniques 
available for this task, e.g., Duncan’s multiple range test [11], the following 
typical variances are given. 
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a. The difference between two over-all means of dimension u, X...., — 
X...... , has a variance which is estimated by 


2 M.S. (Error 1) 
QRST 


The difference between two means of dimension ¢ for a given level of 
dimension u, X...;, — X...:-. , has a variance which is estimated by 


2((U — 1)M.S. (Error 1) + M.S. (Error 2)] 
QRSU 


The variances of differences between means of dimensions r or s for particular 


levels of u take analogous forms. 
c. The difference between two means of dimension u for a particular 
level of t, X...44 — X...1.- , has a variance which is estimated by 


2 M.S. (Error 1) 
QRS 
Variances of the differences between two means of dimension u for particular 
levels of other dimensions take analogous forms. 
d. The difference between two means of dimension t, X_..,. — X....-., 
has a variance which is estimated by 
2 M.S. (Error 2) 
QRSU 
Variances of the differences between means of dimensions r and s take 
analogous forms. 
e. The difference between two means of dimension ¢ for a particular 
level of s, X..... — X...::. , has a variance which is estimated by 
2 M.S. (Error 2) 
QRU 
Variances of differences between means of dimension ¢ for particular levels 
of r and also between two means of other combinations of r, s and ¢ follow 
analogous forms. 














. 





A Four-Dimensional System with Levels of One Dimension Correlated p, and 
Levels of a Second Dimension Correlated pz 


Under different conditions than in the previous section, a change in the 
assumptions made there on variances and covariances seems warranted. 
Suppose that Q different subjects are grouped in each of RS subclasses 
corresponding to dimensions r and s. Now let all QRS subjects be administered 
U levels of dimension wu under all levels of dimension t. Here each subject 
in the rsth subclass is observed under 7'U dimension level combinations. 
Assume that the correlations between observations of levels of dimension « 
made on the same subject for the same level of ¢ differ from the correlations 
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between observations taken under different levels of ¢. For example, the 
levels of dimension ¢ might refer to various points in time at which each subject 
is administered all levels of wu. 

For this situation let the model state that the random variables X,,.:, 
are QRSTU-variate normally distributed with 


(10)  V(Xerets) = 073 


COV(X oretuy Xg’r'e’t'u’ 


=p.o, for q=q’, r=r’, s=s', t=, uxu’, 


ll 
Il 


poo, for q q, r=r, s=s', t#¥?, 


0 otherwise; 
and 
(11) E(X retu) = the linear function given in (2). 
Again transform the variates to independent sets by means of an ortho- 
gonal transformation applied twice. First transform X,,,,, by means of 


Z p Aen 


Ww =] 
Bimre = w + 1 — -- w ? 


(12) (w= 1,2,---,U—1)- 


U 
ey Zs A greta 
¥ wise — VU ges. ane 4 


Thereafter, the Y,,.:7 of (12) are transformed by the same form of the trans- 
formation to Z,,.:v , of which the Z,,.7v are labeled W,,.7v . Resulting from 
these transformations is a set (1) of QRST(U — 1) variates, Y...(u = 


1, 2, --- , U — 1), normally and independently distributed with variance 
o? = o° [1 — p,]; a set (2) of QRS(T — 1) variates, Z,,.:0 (¢ = 1, 2, -+-, 
T — 1), normally and independently distributed with variance 02 = o” 


[1 + (U — 1)p, — Up]; and a set (3) of QRS variates, W,,.ry , which are 
normally and independently distributed with variance o2 = o* [1 + (U — 
1)p, + (7 — 1)Up,]. Again these variates are independent between sets. 

It follows that the sums of squares associated with set (1), set (2), and 
set (3) are given by (138), (14), and (15), respectively. 


1) LODE Lew — Kewl 
z; Pm PF > pe bec sis Mees ey a(4, u) a B(4, ru) 
— B(5, su) — BG, tu) — y(2, rsu) — y(8, rlu) — y(4, stu 


_ ssw | 
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T-1 
ent Ps 2 »» [Zorsey — E(Zareev) 


“UL2E2L2 EE — Fee — 9(8, ) — 62, r) 





— 6(3, st) — y(1, rap | 
(15) b 3 > > M (Worery — E(Worarv)] 


= TU d p> > | See —n-a(l,r) — a(2,8) — attr) | 


As before, tests of hypotheses on parameters involved in the three variate 
sets may be derived. These tests are given below and the analysis of variance 
follows in Table 2. 

In set (1) the test of the hypothesis of no third-order interaction, H, : 
6(rstu) = 0, is accomplished by forming the F ratio 


8.8. [d(rstw) //(R — I(S — I(T — 1I)(U — VD 
S.8.(Error 1)/(Q — 1I)(RST)(U — 1) 


2 
. 





(16) F(6) = 


Other hypotheses in set (1) are tested by forming analogous F ratios. 
Again in set (2) we test the hypothesis, Hy : y(1, rst) = 0, by referring 
the ratio 


* _ §.8. fy, rs) |/(R — 1)(S — 1)(T — 1) 
(17) Fly)! = “g's (error 2)/(@ — I(RS\T — 1) 





to its proper F distribution and other hypotheses of set (2) are tested by 
forming comparable F ratios. 

Finally, in set (3) the hypothesis, H, : 8(1, rs) = 0, can be tested by forming 
the F ratio 


veqyy _, 8:8-(6U, eR — 118 — 1) 
(18) FI8()] = “g's Gevror 3)/(@ — DRS) 





The three remaining hypotheses in set (3) may be tested by forming compar- 
able F ratios. The analysis of variance for this system is given in Table 2. 

The notation used in the sum of squares column of Table 2 has been 
defined in (9), and the expected mean squares due to error are 


E{M.S. (Error 1)} = of = o (1 — p,), 
E{M.S. (Error 2)} o [1 + (U — 1)p, — Up,], 
E{M.S. (Error 3)} = 03 = o° (1 + (U — 1)p, + (T — 1)Upy]. 


2 
2 
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TABLE 2 


Analysis of Variance for a Four Dimensional Layout in Which the 
Observations of Levels of One Dimension Are Correlated 


QP, and of the Levels of Another Dimension Are 


Correlated Po 











Source of — Sum of Square 
Variation Sida — 
Total 

set (1) QRST(U-1) a'-b! 
(Between "u" 

within "t" 

summed over 

individuals) 
a(4,u) U-1 o-p 
B(4,ru) (R-1) (U-2) i-f£-o+p 
B(S,su) (S-1) (U-1) j-m-o+p 
6(6,tu) (T-1) (U-1) k-n-o+p 
y(2,rsu) (R-1) (S-1) (U-1) c-f-1-j+l+m+o-p 
7(3,rtu) (R-1) (T-1) (U-1 d-g-1-k+l+n+o-p 
7(4,stu) (S-1) (T-1) (U-1) e-h- j-k+m+n+0-p 
8(rstu) (R-1) (S-1) (T-1) (U-1) a-b-c-d-e+f+g+h+i+ J+k-f-m-n-o+p 
Error (1) (Q-1) (RST) (U-1) a'-b!-a+¢b 
Total 

set (2) QRS (T-1) b'i-c! 
(Between "t" 

summed over 

individuals) 
a(3,t) T-1 n-p 
B(2,rt (R-1) (T-1) g-l-n+p 
B(3,st) (S-1) (T-1) h-m-n+p 
7(1,rst) (R-1) (S-1) (T-1) b-f-g-h+1+men-p 
Error (2) (Q-1) (RS) (T-1) b'-c!-bef 
Total 

set (3) QRS oil 
(Between 

individuals) 

u 1 P 

a(1,r) R-1 t-p 
a(2,s) S-1 m-p 
B(1,rs) (R-1) (S-1) f-f£-m+p 
Error(3) (Q-1) (RS) c'-f 





The variances of certain typical differences in means follow. 
a. The difference between two means of dimension u, X..... — X....u° , 
has a variance which is estimated by 
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2 M.S. (Error 1). 
QRST 


b. The difference between two means of dimension t, X ...4. — X...1°,, 
has a variance which is estimated by 


2 M.S. (Error 2). 
QRSU 


c. The difference between two means of dimension s, X,.,.. — X...’.. , 
has a variance which is estimated by 


2 M.S. (Error 3) 
QRTU 


Variances of the differences between means of r take analogous forms. 
d. The difference between two means of dimension u for a particular 


level of t, X..... — X......, has a variance which is estimated by 


2 M.S. (Error 1) 
QRS 


Variances of the differences between means of dimension wu for particular 
levels of r and s take analogous forms. 

e. The difference between two means of dimension ¢ for a particular - 
level of u, X...4. — X...4.., has a variance which is estimated by 


2{(U_— 1)M.S. (Error 1) + M.S. (Error 2)]. 
QRSU 

















f. The difference between two means of dimension s for a particular 
level of u, X..,., — X..,*.., has a variance which is estimated by 
2[((U — 1)M.S. (Error 1) + M.S. (Error 3)] 
QRTU 
Variances of the differences between means of r for a particular level of u 
take analogous forms. 
g. The difference between two means of dimension s for a particular 
level of r, X.,,.. — X.,,°., , has a variance which is estimated by 
2M.S. (Error 3). 
QTU 


Variances of the differences between means of r for a particular level of s 
take analogous forms. 








Systems of Lower Dimensionality 


Due to limitations on space it will not be possible to examine special 
cases of the previous systems wherein the number of dimensions is less than 
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four. It is suggested that systems of lower dimensionality be viewed as cases 
of the preceding systems in which certain subscripts, say r, s, or t, have been 
deleted and Tables 1 and 2 be altered to take care of these setups. Thus, all 
parameters in Table 2 containing a deleted subscript will be omitted from 
analysis and the sums of squares and degrees of freedom which remain will be 
altered by striking out either the deleted subscripts or corresponding limits 
of summation as the case might be. Tests of hypotheses are made in the same 
manner as before. 


Problems in Interpretation 


In classification systems, whether the levels of the dimensions are 
experimental or not, the difficulties encountered in interpreting the results 
of an analysis usually increase when the number of dimensions are increased. 
This is particularly true when it is desired to explain the meaning of significant 
higher-order interactions. Methods useful in the explanation of significant 
higher-order interactions (see [12]) for systems in which the observations are 
assumed to be uncorrelated may be used also in the setups presented in this 
paper. 

Normally it is necessary to compare individual means in order to explain 
both significant main effects and interactions, this being accomplished by 
tests of significance such as multiple range tests. In the previously presented 
models, however, this task becomes more complicated since the analyses 
contain more than one error variance. Although the variances of differences 
between particular means which previously have been given should facilitate 
such comparisons, it should be pointed out that the degrees of freedom for 
variances which involve more than one error mean square may be approxi- 


mated by means of methods given by Taylor [16]. 
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THURSTONE’S ANALYTICAL METHOD FOR SIMPLE 
STRUCTURE AND A MASS MODIFICATION THEREOF* 


RoBeErt R. SoKkart 
UNIVERSITY OF KANSAS 


The analytical method for simple structure proposed by Thurstone is 
applied to four separate cases and found to yield satisfactory results. The 
simple structure obtained by Thurstone’s method is found to match closely 
that obtained by other methods and corresponds to the true structure of the 
matrix in those cases where true structure is known. Difficulties about the 
choice of the correct trial vector led the writer to develop a modification of 
Thurstone’s method, useful where high speed computational facilities are 
available. Instructions are given for this so-called mass modification, and 
the procedure is illustrated with a 5-factor, 14-variable example. While the 
results do not fully correspond to a previous graphical solution, it can be 
argued that the results obtained by the new method show an improved 
simple structure. The modified method is applied to three other correlation 
matrices, yielding in each case a satisfactory simple structure. 


The purpose of this paper is: (1) to give several examples of the appli- 
cation of Thurstone’s analytical method for finding simple structure in a 
factor matrix; (2) to compare the results of Thurstone’s solution with those 
from other analytical solutions as well as with results of trial-and-error 
graphic rotation; (3) to introduce a mass modification of Thurstone’s method, 
which is believed to be superior where high speed computational devices 
are available. 

The student or investigator entering the field of factor analysis at this 
time need no longer be dismayed by the prospect of seemingly endless trial- 
and-error rotation of reference vectors interspersed with hours of tedious 
desk calculator computation. A number of analytical solutions for simple 
structure have been proposed by several authors. These methods determine 
a simple structure constellation by arithmetic rather than geometric operations 
and appear to have removed the trial-and-error element from the procedure. 


*Contribution No. 961 from the Department of Entomology, University of Kansas, 
Lawrence, Kansas. 

tMost of the work on which this paper is based was performed during the summer 
of 1956 when the author was the holder of an Elizabeth Watkins Faculty Scholarship 
granted by the Kansas University Endowment Association. The work was carried out 
at the University of Illinois, where the author was ig po to spend the tenure of this 
scholarship. The writer is indebted to Professor L. H. Lanier, the Chairman of the Psy- 
chology Department, who graciously placed space and equipment at his disposal, and to 
Professor Raymond B. Cattell for his continuing encouragement of and interest in the 
author’s work. Some of the computations were performed on the ILLIAC digital com- 
puter. The many courtesies extended by Professor J. J. Nash, Director of the Digital 
Computer Laboratory of the University of Illinois, are gratefully acknowledged. The 
writer is indebted to Mr. John R. Hurley for much assistance during the development of 
the computational routines and for a critical reading of this paper. Expenses in connection 
with the work were met by a General Research Grant of the University of Kansas. 


237 








238 PSYCHOMETRIKA 


Computational labor required for these solutions is generally quite heavy 
but a parallel development, the advent of high speed electronic computers, 
is eliminating this factor for those investigators fortunate enough to have 
access to such a machine. Thus the decision on whether to rotate to simple 
structure may in future be based mainly on the intrinsic merit of that structure 
for a particular research problem rather than on whether an investigator 
has the time or fortitude to undertake a search for simple structure by the 
traditional methods. 

Analytical solutions were published almost simultaneously by a number 
of authors. Carroll [1] proposed a solution based on the minimization of the 
sums of the cross products of squares of factor loadings. This method yields 
orthogonal or oblique factors, as desired. Saunders [8] published a solution 
based on a maximized kurtosis of factor loadings but with factors restricted 
to orthogonality. Pinzka and Saunders [7] extended this solution to the 
oblique factor case (oblimax). Neuhaus and Wrigley [6] developed another 
method (quartimax), based on maximizing the sum of fourth powers of factor 
loadings. The equivalence of these methods in the case of orthogonal factors 
has been shown by Ferguson [3] working on an objective definition of 
parsimony in factor analysis. Tucker’s method [13] is a computational 
routine searching for linear constellations of test vectors. 

Thurstone’s analytical method [12], for simplicity’s sake hereafter 
referred to as TAM, uses a set of arbitrary weights which adjust trial vectors 
into positions perpendicular to such hyperplanes as exist in the study. The 
method is briefly reviewed in the following section. Except for the example, 
discussed in Thurstone’s paper [12], the writer has been unable to find 
published examples of the application of TAM.* Of the several methods 
listed above, TAM appeared the simplest from the point of view of using a 
mixed desk-calculator, digital-computer routine. Before applying TAM to 
his own data the writer tried it out on several other matrices and compared 
the results of his efforts with either the known structure of the matrix or 
with results obtained by other methods, as the data permitted. These findings 
are presented below in the hope that they may prove of interest. 

Several complications arose during the application of Thurstone’s 
original method to some of the matrices; the writer subsequently developed 
a modification of TAM, which he believes is superior when high speed compu- 
tational facilities are available (mainly for matrix multiplication and inver- 
sion). The method is described in the last section of this paper and four 
examples of its application to actual cases are given. 

*A mimeographed report by Rolf Bargmann, entitled “‘A comparison of new ana- 
lytical methods for the determination of simple structure,’’ September 1953, was brought 
to the notice of the author after this manuscript had been completed and submitted for 
publication. Bargmann applied TAM to the 21-variable, 8-factor Fo matrix from p. 402 
of Thurstone’s Multiple-Factor Analysis. A satisfactory simple structure resulted. Several 


improvements of TAM based on Thurstone’s single planing method were suggested by 
Bargmann in his report. 





| 


eee. Se ee: ed 








ROBERT R. SOKAL 239 


Brief Review of Thurstone’s Analytical Method (TAM) 


TAM starts with an unrotated centroid factor matrix F, of dimensions 
n Xk, representing n variables and k extracted factors. A trial vector is 
chosen from one of the variables of the study, and the row of factor loadings 
of variable 7 in matrix F, is transposed into a column vector J; of direction 
numbers. Column vector J; is then normalized to yield column vector P; , 
which changes the direction numbers into direction cosines. 

Next the column vector of projections v;, of normal-length test vectors 
j on unit trial vector P; are computed, 


Vip = F.P;. 


A weight w,, is assigned to each v,, according to the table of weights given 
in Thurstone [12] or, in some cases to be described below, according to a 
scheme developed separately for each reference vector to be determined. 
These weights form the principal diagonal of a diagonal matrix W of order 
n. A weighted factor matrix F,, , 


F, = WF., 
is computed and a symmetric matrix A is then obtained, 
A = FYF.. 


This symmetric matrix is of order k. The next step is the computation of 
A~*, This matrix is used to compute column vector U, 


U = A'P,. 


U is normalized to yield A, , the direction cosines of the first reference vector. 
The column vector A, defines the adjusted position of the initial trial reference 
vector P; . Loadings of the n variables on the new reference vector Vj, 
are computed by 


U54 = FA. 


A second trial vector is chosen from among variables that are in the 
hyperplane of the first reference vector, a third from those in the hyperplanes 
of both the first and the second, and so forth. Occasionally convergence 
toward the simple structure solution may be slow, and it will be necessary to 
iterate the procedure using A, as the new unit trial vector. In practice we 
start with V;, in place of V;, , weighting the former, and continuing as before. 

As will be seen below a trial vector may at times be adjusted into a 
reference vector that has already been determined (a disheartening experi- 
ence in the original TAM since it does not become apparent until the 
tedious computation of the inverse matrix has been completed). This was 
one of the causes leading up to the development of the mass modification. 
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Examples of the Application of Thurstone’s Analytical Method 
The Cattells’ Boys 


The first factor matrix tested is an artificial one simulating the results 
of manipulation tests of seven mechanical puzzles (variables) where success 
is determined by three factors. It was developed by Cattell and Cattell [2] 
in connection with their method of parallel proportional profiles. They 
prepared two similar matrices, one to represent the factor structure of boys, 
the other that of girls. Only the former was employed in the present study. 

The unrotated factor matrix V,, ({2], Table IIIT) was used as the initial 
F, matrix. Variable 7 was chosen as the first trial vector. This yielded the 


TABLE 1 


Comparison of Simple Structure Solutions for Cattells' Boys: 
TAM, Graphic Rotation, and Original (Known) Factor Matrix 

















Reference eg Original TAM Graphic 
vector Variabl rotation* 
I 60 64 65 
2 -50 -43 “45 
3 -10 -14 “14 
4 00 -06 00 
F 90 92 ge 
6 -60 -58 ~65 
: 05 06 -O1 
II EE -40 -32 -35 
2 -80 -84 -83 
3 80 79 80 
4 50 ke 45 
5 -10 06 O01 
6 00 -01 -01 
7 00 10 06 
III a: -10 -08 -06 
2 10 05 11 
3 00 02 -O4 
4 -60 -59 -63 
5 00 08 08 
6 70 67 67 
a 80 80 81 





* Readers who compare these figures with those in Cattell and Cattell [27 
will note that the loadings of variables 2 and 3 for all three reference 
vectors have been interchanged. This is due to a printer's error in the 
left half of table VI of that paper. The present version is the correct 
one according to a personal communication of Prof. R. B. Cattell. 
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Cattells’ reference vector III. Variable 3 had the lowest loading on that 
reference vector and was used as the second trial vector yielding in turn 
reference vector II of the Cattells. The final trial vector was clearly variable 
5, since it was the only variable located in the hyperplanes of both previous 
RV’s. It yielded the first RV of the previous authors. The weights used were 
the original ones tabled by Thurstone ({12], Table 1). Since the initial factor 
matrix was specially designed to represent simple structure and consequently 
exhibited a wide range in the magnitude of the factor loadings it is not sur- 
prising that the original weight scale was adequate. 

The result of the above procedure is shown in Table 1. The graphic 
solution and TAM are almost identical and both closely resemble the original 
structure of the factor matrix. Correlations between the TAM reference 
vectors are very low (rz.77 = .136, rrr = .096, riz.r27 = .061) returning 
almost completely to the initial orthogonal position. In this particular 
jnstance TAM appears to give entirely satisfactory results. 


The Johnson-Reynolds Data 


The next example is a small matrix which had been used previously by 
two other workers in demonstrating their own analytical solutions for simple 
structure (Carroll [1], Pinzka and Saunders [7]). The matrix had been originally 
computed from ten verbal tests administered to 113 students by Johnson 
and Reynolds [5] and contains 10 variables (the tests) and two factors. 

TAM was applied to the factors using variable 5 as a first trial vector. 
This resulted in a factor similar to factor I of Carroll and Saunders. Since 
the hyperplane was determined by only one variable the problem was iterated 
using the V;, column as a new V;, column. The second factor was found via 
variable 2 which lay in the hyperplane of factor I. Thurstone’s weights 
({12], Table 1) had to be modified somewhat since their original magnitudes 
would not have given enough differentiation. Thurstone’s suggestion ({12], 
p. 180) that the total range of v;, values be divided into a number of equal 
intervals and that these intervals be assigned values of from 0 for the highest 
absolute loading to 6 for the lowest such loading was found to be quite 
effective. In cases such as Wright’s chicken bones discussed below, the v;, 
range was not such as to warrant division into seven classes, and weight 
classes of from 0 to 4 were assigned this and similar cases. 

Table 2 compares the findings by TAM with those of Carroll and 
Saunders [1, 7]. It can be seen that the solution by TAM is quite similar to 
that by the previous workers. The iteration of factor I brought the loadings 
for that factor considerably closer to the loadings found by the other two 
methods. Unless one can agree on a common optimal criterion of simple 
structure it is impossible to compare and evaluate the three solutions. Pinzka 
and Saunders [7] show that their solution is the better by their criterion while 
Carroll’s is the better by his criterion. The f value (Pinzka and Saunders’ 
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TABLE 2 


Comparison of Three Analytical Methods for Simple Structure 
Applied to Johnson-Reynolds Data 


Reference Pie aay Carroll Pinzka and TAM TAM 
vector Variable Saunders (lst iteration) (2nd iteration) 














I 1 -232 -2h2 -119 -22 
2 -099 -108 007 -091 
3 064 056 1k9 070 
4 315 «+ 303 440 325 
5 638 627 739 646 
6 608 597 712 616 
7 629 620 715 636 
8 574 568 638 580 
9 578 568 681 587 

10 281 267 426 292 
II 1 538 532 540 
2 436 427 439 
3 265 254 269 

4 256 234 265 5 

° 

5 -017 -O44 -006 8 

6 010 -017 021 g 

7 -066 -092 -056 a 
8 -113 -136 -105 
9 021 -005 032 
10 350 327 359 





criterion) for the TAM solution is .0495, the worst of the three..On the other 
hand the correlation between the reference vectors is — .7058 for Pinzka and 
Saunders’ solution while it is only — .4862 for TAM. Without prejudging 
the other analytical solutions it may be said that TAM did provide a satis- 
factory solution in this instance. 
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Wright's Chicken Bones 

This is a biological correlation matrix based on six measurements of the 
bones of 276 chickens. These data were first published by Wright [14], who 
analyzed them by a method analogous to factor analysis. His findings were 
revised in a later paper [15]. The correlations appear to be accounted for by 
three common factors [10, 14, 15]. These data have been rotated to simple 
structure from a centroid factor matrix twice independently by the writer 
and his associates. The first two columns of Table 3 show these independent 
solutions. A uniform numbering system has been adopted for the reference 
vectors. Reference vector I appears to represent a head factor, while reference 
vectors II and III clearly represent wing and leg factors, respectively. From 
Table 4 which shows the correlations between reference vectors we note that 
the wing and leg factors are almost collinear. 

The analysis by TAM presented some difficulty. The use of Thurstone’s 
original weights with variable U resulted in an essentially unrotated but 
reflected factor III rather than in factor II as expected. This difficulty could 
be overcome by recoding the weights based on the range of factor loadings 
as discussed in the previous section. An alternative procedure in this six- 
variable case was simply to array the v;, values by order of absolute magnitude, 
assign weight 0 to the largest value and 1 to 5 to values of decreasing magni- 
tude. 

When these revised weighting techniques were applied to the V;, column 
vector obtained from variable U, the resultant A matrix could not be inverted 
during a first attempt on a desk calculator, using the cutomary Dwyer 
technique and carrying four decimal places. Investigation disclosed that the 
determinant of the A matrix was close to zero (| A | = .013), apparently due 
to the high correlation between the wing and leg factors. When the inversion 
was repeated on the ILLIAC in 12-decimal arithmetic, a solution was found 
which finally yielded reference vector II of the graphic solutions. 

The next variable chosen was F, which is in the hyperplane of reference 
vector II. It yielded previous reference vector III. Again the determinant 
of the A matrix was very small requiring five-decimal arithmetic. The final 
trial vector was based on variable B which had been in the hyperplane of 
both reference vectors found so far. This yielded reference vector I of the 
previous analyses. In view of the low correlation of this reference vector 
with the other two, the determinant of the A matrix was considerably larger 
than zero, and its inversion proceeded without difficulty. 

A glance at Table 3 will convince the reader of the similarity of TAM 
with the two graphic solutions. Table 4 shows the similarity of the three 
solutions in terms of the correlations between pairs of reference vectors. In 
spite of initial difficulties TAM again produced essentially the same simple 
structure as the graphic method. 
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TABLE 3 


Comparison of Simple Structure Solutions for Wright's Chicken 
Two Graphic Solutions and TAM 


Bone Data: 

















Reference rs Graphic Graphic TAM 
vector Variable #1 #2 

I t 4g 47 48 

B 50 54 hg 

H 09 05 08 

U 02 02 00 

F Ol 03 -O1 

T 06 08 o4 

II 2 1 09 03 

B -04 -01 -02 

H 27 32 29 

U 33 33 35 

F -O01 Ol Ol 

Tr -02 01 00 

III L -O4 00 -02 

B 00 o4 O01 

H “O01 10 02 

U -04 10 -O1 

F 30 41 33 

T 29 ko 32 





Explanation of variables: 


length, U = ulna length, F = femur length, T = tibia length. 


L = head length, B = head breadth, H = humerus 


TABLE 4 


Correlations between Reference Vectors 
of Simple Structure Solutions of Table 3 








Correlation coefficients 








Method 

1°ad I*III II<iit 
Graphic #1 -.22 Sued - .86 
Graphic #2 -.22 -.18 -.82 
TAM -.21 -.21 -.05 
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Stroud’s Termite Soldiers 


These data, published by Stroud [11], are based on a correlation matrix 
involving 14 physical measurements of soldiers of 48 species of the termite 
genus Kalotermes. The correlations were factored by the centroid method; 
the unrotated F, matrix is given in Table II of Stroud’s paper. Five factors, 
extracted from the correlations, were rotated by a trial-and-error graphic 
method. The rotated solution is given in Table V of Stroud [11]; the plots 
of pairs of reference vectors are in Figure 3 of the same article. 

The writer obtained only one reference vector from these data by TAM. 
During the analysis of this example the idea of a mass modification came 
to the writer, and it was decided to carry out the mass analysis of Stroud’s 
data reported in this paper. The single trial vector used was based on variable 
12, following Thurstone’s [12] instructions that the first trial vector should 
be based on a test (variable) which has some relatively high correlations in 
the correlation matrix and which also has an appreciable number of relatively 
low coefficients. However, the reference vector obtained showed only a 
poorly defined hyperplane and did not match any of the reference vectors 
found by Stroud. This would appear to be the first failure of TAM to yield 
a reference vector corresponding to one obtained graphically during a first 
trial. A discussion of this case will be deferred until the discussion of the 
analysis of the Stroud data by the mass modification of TAM. 


A Mass Modification of Thurstone’s Analytical Method 
Description of the Procedure 


The mass modification of TAM was developed in response to several 
problems which arose during the application of this method to the matrices 
of the preceding section. Some of these problems were the following. 

(1) Occasionally a given trial vector would not yield a reference vector 
with a well-defined hyperplane. 

(2) At other times a given variable would result in a well-defined refer- 
ence vector, but a second variable would have given an even clearer simple 
structure. It might be argued in such a case that iteration of the reference 
vector from the first variable would also have yielded better definition. 
However, it is of course desirable to limit the number of iterations to an 
essential minimum. 

(3) At times the prescribed procedure for finding new trial vectors in 
TAM produced two collinear reference vectors and lost one of the significant 
reference vectors in the study. This particular topic is discussed further in 
Stroud’s termite soldier example below. 

The essentials of the mass modification of TAM, which will subsequently 
be referred to as MTAM, are the simultaneous use of every variable as a test 
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vector. Thus from an n X k unrotated factor matrix, F, , where n is the 
number of variables and k is the number of factors, n trial vectors and con- 
sequently n reference vectors are computed, which however represent only 
k dimensions. The advantages of this method are as follows. 

(1) A uniform treatment of every factor matrix without specific decisions 
to be made before each reference vector is established. 

(2) The best definition for a given reference vector can be chosen from 
among those provided by several trial vectors. 

(3) A choice can be made about the obliqueness of reference vectors 
since a A matrix for all reference vectors will be obtained. 
The advantages of MTAM are, however, predicated upon the availability 
of high speed computing machines since they involve for each iteration the 
computation of n inverse matrices of order k. 

The computational procedure for MTAM is given below. 

(1) The procedure starts with the unrotated factor matrix F, of order 
n X k. 

(2) F, is normalized by rows to Fy . This gives all possible unit trial 
vectors P, . 

(3) Compute 


r i) 
V;, = F.Fx, 


where V;, is of order n X n. This yields all possible column vectors v;, . Each 
column of V;, is a column vector »;, . 

(4) Weights are assigned separately to each v;, based on the magnitude 
and range of the v,;, values as described above. A routine procedure which 
xan be given to a clerk or built into a computer routine can be easily set up. 
The weights are set up as n X n diagonal matrices W, through W, . 

(5) Matrices W, through W,, are postmultiplied by F, . There are, of 
course, ” such multiplications, which yield n matrices (of order n X k) 
Fy, through Fy, . In general 


Py; = W.F.. 
(6) Premultiply Fy; by constant matrix F} to give n symmetric matrices 
of type A; (of order k X k). 
A; = FUF,,. 


(7) Compute inverses A;* for n matrices A, . 

(8) Split F4 into n column vectors of type F%, . 
Then postmultiply the n inverse matrices by the corresponding column 
vectors F'4,, to obtain n column vectors U, , 


r — -1 ATA 
U; nied A; Nie 





al 
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(Some workers with TAM have preferred to solve the simultaneous equation 
AU = F*%, for U instead of the two-step procedure originally formulated by 
Thurstone and followed here. The choice is entirely one of convenience 
and availability of programs for a given computer.) 

(9) Assemble the n column vectors U; into a single matrix U and 
normalize by columns to matrix A (of order k X n). 

(10) The final step consists of computing the n X n reference vector 
matrix V;, , 

V; = F gAs 


The writer performed these procedures step by step simultaneously for 
seven different factor matrices (in about 3 hours computer time plus 3 days 
handling time, discounting errors made as the technique was being perfected). 
However, it would seem preferable to program the entire sequence as a 
single routine. This would then permit iterations. Machine output should 
include V,;, and A (or preferably A’A). In the case of matrices of appreciable 
size, such as 50 tests (variables) and 10 factors, storage facilities on most 
computers would be inadequate to handle all the steps in a single procedure. 
In such a case the n X n weight matrices W, through W, would have to be 
entered singly into the machine and steps (5) through (8) would have to be 
performed in sequence separately for each trial vector. This would also permit 
dropping out any trial vector if the operator so desired. Subsequent procedures 
for finding optimal simple structure are discussed in the next section in 
relation to a concrete example. 


The Mass Modification Applied to Stroud’s Data 


The 14-variable, 5-factor matrix of termite soldier measurements by 
Stroud [11] mentioned previously was subjected to the MTAM procedure 
just described. The matrix was completely processed once as an experimental 
problem to study the feasibility of MTAM. After the method had been 
found to be effective new weights were assigned to the resulting V;, matrix 
and the problem was iterated. Table 5 shows V;, of this second iteration. It 
gives the factor loadings (correlations with the reference vectors) based on 
the fourteen trial vectors corresponding to the fourteen variables. Table 6 
shows the correlations between these fourteen reference vectors calculated by 


Ce = APA. 


The incumbent problem is to select from the fourteen reference vectors 
five which will yield a satisfactory simple structure. At the end of an MTAM 
analysis one is, of course, able to carry out very simply and elegantly 
Thurstone’s [12] original procedure of starting out with a likely trial vector; 
then using a variable in the hyperplane of the first, followed by a variable 
in the hyperplane of both reference vectors, and so forth. All that is needed 
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TABLE 5 


Vin Matrix (Correlations of Variables with Reference Vectors) 
after Second MTAM Iteration to Stroud's Centroid 
Factor Matrix for Termite Soldiers 











Reference vectors 








Variables 

a ee he ee Re oe ee OS ee ee 

1 32 11 O7 21-10-12 11 02 OF #17 #09 OF 26 28 

2 14 57 52 08 -11 -05 Ob 02 -05 O8 -08 16 O7 -02 

3 00 62 64 -06 08 15 -08 OO 05 -03 05 -12 -10 C6 

4 23 01-01 33 20 16-17 -08 -10 32 -c8 oOo 31 OF 

5 -07 -10 -05 -10 45 42 -26 06 07 18 12 -Ol Oh -O1 

6 -O7 14 17-14 47 4&7 +34 -01 -0% 21 -01 02 05 -10 

7 07 OO -O1 -O4 -07 -08 23 30 34 -06 36 17 02 16 

8 -07 06 Ob -06 03 OF 15 36 29 -09 29 36 -Oh -08 

9 05 -Ol 02 -06 O01 00 18 26 37 -06 41 OL -02 25 

10 21 00 -O1 -30 22 18 -17 -06 -07 312 -Oh Ob 2 11 

ll 07 -03 00 Ol 11 09 06 17 27 Ob 32 -Oh Ob Ob 

12 08 -01 -08 18 03 O1 OF 18 02 12 OO 49 19 -18 

13 23 -10 -12 25 10 05 -03 02 OF 2h oF, c&8 2 18 

14 2h 02 03 10 -03 -05 07 -03 13 11 18 -2h 14 40 
Number of 
variables 


in hyperplane 8 8 10 6 6 8 6 9 8 6 8 8 7 5 





is to look up the values of v;, in V;, . However, the original TAM which 
proceeds somewhat blindly from reference vector to reference vector will 
soon fail to appeal to the experimenter. A multiplicity of possible RV choices 
becomes available. Some result in five reference vectors, others only in four. 
Thus the outcome of TAM is to some degree dependent upon the choice of 
an initial variable. Having gone through the MTAM computational pro- 
cedures the experimenter has at his disposal considerable added information, 
including the number of variables in the hyperplanes of all possible reference 
vectors as well as all mutual correlations between these vectors. It should 
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be possible from this information to develop a procedure leading to a unique 
and optimal set of reference vectors. 

Considerable time and effort have been spent by the writer on developing 
a simple “cookbook” procedure for obtaining an optimal simple structure 
using V;, and Cz of MTAM. Regrettably, no such simple routine valid for 
all investigated examples could be devised. However, a series of steps may 
be outlined which in the cases tested has yielded satisfactory results. 

(1) Examine Cp (and also check V,, and A) for pairs or groups of highly 
correlated (| r | >.8) reference vectors. These are usually based on closely 
related variables (in psychological matrices tests measuring nearly identical 
psychological functions). Some method of cluster analysis of Cz is helpful 
in this procedure. In the matrix of Table 6 the following nonoverlapping 
clusters could be isolated considering only correlation coefficients larger than 
0.8: 2-3, 5-6-7; overlapping clusters: 1-4-13, 4-8-9-10-11, 8-13, 10-13. 
The variables in a given cluster usually represent a certain reference vector. 
The choice of which variable is to represent the factor is made at a later 
stage, except in cases such as those described below where the reference 
vectors in a cluster are essentially collinear (e.g., reference vectors O and P 
in the housefly data were correlated at r = .999). In such cases it is, of course, 
immaterial which reference vector is used to represent the cluster. 

(2) Count the low correlations (| r | < .2) of each reference vector with 
other reference vectors (see Table 6). Below is shown the code number of 
each reference vector followed by the number of vectors with which it shows 
low correlations: 


j-] 8-0 
2-5 9-3 
3-4 10-0 
m 4-1 11-4 
o-3 12-2 
6-3 13-1 
7-3 14-2 


In some cases, such as Wright’s chicken bones discussed above, correlations 
as low as .2 do not exist. In such circumstances the definition of a low correl- 
ation must be altered to permit the formation of groups of relatively un- 
related reference vectors. 

(3) Count the number of variables in the hyperplane of each reference 
vector (factor loadings between —.10 and +.10.) This has been done for 
the Stroud matrix at the foot of Table 5. 

Now proceed to select the reference vectors for a simple structure in 
this matrix. Our eventual criteria for simple structure will be the five requisites 
listed by Fruchter ([4], p. 110), namely: 

(a) Each variable should have at least one loading close to zero. 
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(b) There should be, for each factor column, at least as many tests with 
zero loadings as there are factors. 

(c) For every pair of factors there should be several variables with 
projections on one factor vector but not on the other. 

(d) For problems having four or more factors, a large proportion of 
the variables should have negligible loadings on any pair of factors. 

(e) Only a small number of variables should have appreciable loadings 
on any pair of factors. 

However, attention is first directed to securing a large number of zero 
or near zero loadings for each reference vector adopted and low correlations 
between reference vectors whenever possible. 

Array all of the reference vectors in accordance with their suitability 
with respect to the number of variables in the hyperplanes of the reference 
vectors: 3, 8, 1-2-6-9-11-12, 13, 4-5-7-10, 14 (cf. bottom line of Table 5). 
When examined for the largest number of low correlations with other refer- 
ence vectors the following array by suitability resulted: 2, 3-11, 5-6-7-9, 
12-14, 1-4, 13, 8-10 (ef. Table 6). 

Checking back with the high correlation clusters of reference vectors, 
only one representative each of the following two clusters should be chosen: 
2-3 and 5-6-7. Reference vector 2 was chosen over 3 arbitrarily since the 
two are almost collinear. Reference vector 6 seems slightly more suitable 
to represent 5-6-7 since it has two more variables in its hyperplane than 
5 or 7, Reference vector 1 was chosen to represent 1—4—13 by similar reason- 
ing. In cluster 4-8-9-10-11 the choice fell on 11 as the best representative 
by both criteria rather than 8 which, while better by the first array, was 
much worse by the second. Variables in the remaining overlapping clusters 
(8-13, 10-13) having been represented already, the fifth reference vector 
was chosen from variables 12 and 14 which had not entered into any cluster. 
These two reference vectors were quite highly correlated with each other 
(r = —.72); 12 was chosen over 14 by referring to the suitability of arrays. 
It is admitted that this procedure lacks a unique solution, but the sub- 
jective decisions required are usually between quite similar alternatives. 
Thus the result would not have been appreciably different if reference vector 
5 had taken the place of 6, 3 had been in place of 2, 8 in place of 11, ete. 

Table 7 shows the simple structure reference vector matrix adopted by 
graphic methods. Only reference vectors 2 and 6 are clearly identical to 
Stroud’s E and D, respectively. Reference vector 12 is a poor representative 
of Stroud’s B, while vectors 1 and 11 together determine the covariance 
accounted for by Stroud’s A and C. The question may therefore be raised 
whether MTAM has achieved its goal in view of discrepancies with Stroud’s 
solution. However, the reader may convince himself by inspecting Table 7 
that the MTAM rotated reference vector matrix shows excellent simple 
structure. Plots of the reference axes against each other (not shown) confirm 
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TABLE 7 


Stroud's Termite Soldiers: MTAM Solution compared with 
Stroud's Graphic Solution 























MTAM 
Variables Reference vectors 
(1) (2) (6) (11) (12) 
1 32 FER -12 09 ok 
2 14 57 -05 -08 16 
3 00 62 15 05 -12 
4 23 01 16 -08 08 
5 -07 -10 ko 12 -01 
6 -07 14 47 -01 02 
7 07 00 -08 36 17 
8 -07 06 o4 29 36 
9 05 -01 00 kl Ol 
10 21 Ol 18 -04 o4 
11 07 -03 09 32 -O4 
12 08 -01 Ol 00 kg 
13 23 -10 05 07 08 
14 2k 02 -05 18 -2h 
Stroud's graphic solution 
Variables Reference vectors 
A B c D E 
1 36 01 30 07 10 
2 00 06 26 -06 57 
3 00 -04 -01 09 52 
4 19 -05 32 22 05 
5 ol, 03 -03 40 -12 
6 -06 -01 o4 46 12 
7 25 26 o4 -1l -0k 
8 03 38 02 -01 -06 
9 30 18 -06 -04 -10 
10 20 -05 28 2k 02 
1 28 10 -02 -07 -1l 
12 -01 28 30 ok 09 : 
13 28 03 25 10 -08 
14 40 -11 08 -03 -07 





' these conclusions. The MTAM solution has a greater number of variables 
in the hyperplanes and the reference vectors are slightly less correlated (see 
Table 8) than they are by Stroud’s findings. Although this is not the place 
to discuss the findings in detail it may be added that the new simple structure 
is somewhat more interpretable biologically speaking. 

Finally it may be appropriate to mention a supposition of the author 
which he has so far been unable to test. If each matrix were iterated a number 
of times until stability of V;, were reached, it may well be that the n refer- 
ence vectors would have collapsed into a number of positions corresponding 
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TABLE 8 


Correlations between Reference Vectors Of Table 7 














MTAM 
Reference vectors 
1 2 6 11 12 
1 1.00 
2 -.39 1.00 
6 - 47 3 1.00 
11 -.51 -.03 ~.28 1.00 
12 acto 33 -.18 .18 1.00 





Stroud's graphic solution 
Reference vectors 








A B C D E 
A 1.00 
B - 47 1.00 
C 43 -.53 1.00 
D -.38 -.49 Ol 1.00 
E -.62 26 .05 03 1.00 





to the number, k, of factors in the matrix. In such a case the choice of the 
correct reference vectors would, of course, not present any problem and 
advantages (2) and (3) of MTAM given above would vanish. Until MTAM 
has been programmed on a computer (to permit rapid iteration) this suppo- 
sition cannot be tested. The single iteration performed in connection with 
the MTAM study of the Stroud data yielded neither conclusive nor even 
suggestive evidence on this point. 


The Application of MTAM to Three Other Matrices 


Graphic solutions for simple structure are unavailable for these matrices, 
so that the success of MTAM has to be judged by the criteria for simple 
structure mentioned above. The biological plausibility of the structure 
obtained is not discussed, being beyond the stated purpose of this paper. 
These matrices and others will be discussed in detail in several publications 
in preparation. 
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TABLE 9 


MTAM Solution for Correlation Matrix of 14 Morphological Characters 
in Houseflies. Simple Structure Matrix Above, 
Correlations between Reference Vectors Below 








Reference vectors 














I II III Iv Vv VI 
A 34 05 19 -03 32 oh 
B 46 12 00 Ol 25 -05 
Cc 48 -03 41 00 -09 Ol 
D 4b 17 19 -O1 -03 03 
E 12 30 17 03 -03 -O1 
F 2 45 -03 -04 -05 -02 
G -Ol 00 40 00 08 -05 
J -04 03 -04 61 -02 05 
K 03 -04 02 60 00 -05 
L -04 08 22 -01 26 05 
M 09 07 16 10 27 o4 
N 06 -07 17 -04 Ly -03 
0 05 -05 -01 Ol 02 TT 
F -04 o4 00 -01 -03 78 
Reference Reference vectors 
vectors I II Fi ok A IV V VI 
I 1.00 
II -.32 1.00 
III -03 - .67 1.00 
IV .05 -.06 .02 1.00 
Vv .02 -.39 .12 -.02 1.00 
VI -.15 02 -.05 -.17 -.13 1.00 





Table 9 is based on an MTAM analysis of an orthogonal factor matrix of 
six factors and fourteen morphological characters in 421 houseflies, Musca 
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TABLE 10 


255 


MTAM Solution for Correlation Matrix of 18 Morphological Characters 


in the Aphid, Pemphigus populi-transversus. Simple Structure 
Matrix Above, Correlations between Reference Vectors Below 











Reference vectors 

















Variables 
I II III IV V VI VII 
B 25 28 ok 13 o4 -02 -06 
Cc ko 02 07 o4 -05 -01 -17 
D -01 53 06 08 -1l -04 10 
E 37 34 -06 07 oh 02 ok 
F 17 46 -03 -O4 -01 -04 -01 
re! 28 15 05 27 05 -07 -05 
H 06 28 06 16 13 08 -06 
I 09 02 -02 34 -01 -11 02 
L 02 -04 43 07 37 -1 00 
M 08 o4 54 Ol 09 -08 -07 
N 03 -03 21 34 -08 -01 11 
(0) 02 -03 -O4 48 -05 -07 ho 
P -06 -01 08 -09 43 02 O4 
Q 06 00 21 -05 17 07 12 
R -10 01 05 10 -15 32 Oh 
s 08 -01 -09 -02 07 52 -02 
T -12 00 -03 07 20 =08 14 
U “19 07 -06 08 14 -01 32 
Reference Reference vectors 
vectors I II III IV Vv vI VII° 
I 1.00 
rr -.22 1.00 
III -.19 -. 1.00 
IV 12 -.66 -.27 1.00 
V -.18 -.02 10 -.22 1.00 
VI 14 .00 -.26 14 -.15 1.00 
VII -.56 33 -.12 30 23 -.13 1.00 





domestica. The solution appears to be a satisfactory simple structure as seen 
from the factor matrix in the upper part of the table and the correlation 
matrix between reference vectors in the lower part. The simple structure is 
interpretable from the biological point of view. 

Table 10 shows the result of MTAM on an orthogonal factor matrix of 
seven factors and eighteen morphological characters in 343 aphids of the 
species Pemphigus populi-transversus. The original correlations were product 
moment coefficients based on covariances within clones (galls). Again the 
solution appears to be a satisfactory simple structure both from the statistical 
and biological points of view. 
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TABLE 11 


MTAM Solution for Q-Type Correlation Matrix of 4 Genera of Bees 
in the Tribe Osmiini. Simple Structure Matrix Above, 
Correlations between Reference Vectors Below 








Reference vectors 





Variables 














(species 
code 
numbers ) I II III IV 
4 64 06 00 «15 
5 66 -06 -03 09 
8 62 -02 Ol 12 
26 07 22 -09 45 
35 -03 -02 12 
36 00 82 05 -07 
40 -06 -04 -02 70 
90 08 -03 26 41 
o7 -01 ol 88 -O1 
65 -O1 Ol 88 -02 
Reference Reference vectors 
vectors 
I II i IV 
I 1.00 
II @.32 1.00 
III -.3h - 30 1.00 
IV -.33 -.01 -.10 1.00 





A final example is shown in Table 11. This is a matrix based on corre- 
lations between ten species representing four genera of solitary bees of the 
tribe Osmiini. The Q correlations were based on measurements of 122 variables. 
The simple structure obtained by MTAM is very evident. The reference 
vectors represent genera of these bees and the correlations between these 
genera. The primary loadings and correlations between primary factors 
based on the MTAM solution of this matrix have been published elsewhere 


[9]. 
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If we may generalize from the four matrices described above, MTAM 
appears to provide entirely satisfactory simple structure solutions with a 
minimum of (subjective) decisions required from the investigator. 
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ATTENUATION AND INTERACTION 


Quinn McNEMAR 


STANFORD UNIVERSITY 


A significance test is proposed for determining whether a correlation coefficient 
is less than unity by an amount greater than that attributable to errors of 
measurement. 


It is the purpose of this note to point out a connection between correc- 
tion for attenuation and interaction in the analysis of variance. Let X;;n 
stand for the mth parallel measure of the 7th individual on the ¢th test, with 
t= 1,---,N;t = 1,2; and m = a, b. In order to give the problem an analysis 
of variance setting it is necessary to have all scores in comparable form. A 
common metric will result if, by using the respective means and sigmas, all 
four sets of scores are separately transformed to standard scores, with mean 


TABLE 1 


Score Schema 


























Test or 
1 2 
Measure Measure 
a b Mean a b Mean 
Individual (u) (v) (w) (x) (y) (z) 
; Xita Xiib al | a FPN Xi 2b Xi2 
2 Xela Xai Xo. Xo2a X 2b X 22. 
i Kila Xiib Bia, M35 Xian iz 
N Xia XNib Xn. Xn2a Xn 2b Xn2 
Mean Pte oO Pg Oe ee ee ae 
8 2 Oe 8 ee ee ee ae 
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of zero and variance of unity. Thus regard X,,,, as a standard score. These 
necessary transformations, which will not lead to any loss in generality, will 
permit certain simplifications in the sequel. 

The 4N scores, along with possible means, may be set forth as in Table 
1, from which it is readily seen that the setup corresponds to a three-way 
analysis of variance. Keeping in mind that with standard scores certain 
means (as indicated in Table 1) will be zero, we may write out the analysis 
of variance depicted in Table 2. The degrees of freedom add to 4N — 4; 
that this is correct may be inferred from the fact that for each of the four 


TABLE 2 


Analysis of Variance 














Source Sum of squares d.f. Var. est. 
Individuals 42(X, 2 N-1 s?. 
i ey 
Tests Vanishes ¥) -- 
Measures Vanishes 0 -- 
T=xM Vanishes 0 — 
x -X 2 x 2 
Ex T rn, xX, ) N-l “it 
Ix M <<. ¥ N-l s?, 
imim i. im 
Ix TxM nk. =. -— «4. 5 N-l s?. 
re* itm at. i. m itm 
Total pt | Oe 4N -4 
, itm 
itm 





sets of scores the mean is forced to be zero, hence there are four linear re- 
strictions on the deviations entering into the total sum of squares. 

Now if true scores were available on the two tests and if the true scores 
were perfectly correlated, the individual by test interaction would be exactly 
zero. With fallible scores the correlation would be attenuated; that is, the 
correlation between X;,, and X;., would be less than unity by an amount 
attributable to error (of measurement), hence the J X T interaction would 
not be zero. Significant J X T interaction, when tested against the J X T K M 
interaction as error, would mean that the correlation corrected for attenuation 
is significantly less than unity, whereas insignificant J X T interaction would 
lead to the acceptance of the hypothesis that the two tests measure identical 
functions. 








QUINN MCNEMAR 261 


For the tedium of converting to standard scores and doing the required 
computations for an analysis of variance, the tedium of computing correlation 
coefficients, some of which would ordinarily be needed anyway, may be 
substituted. The a and b measures on each test may be regarded as scores 
on parallel halves of the tests, hence each r,, would be a split half reliability 
coefficient (not stepped up), and the correlation between the averages, 
X;,, and Xj. , will be precisely the correlation between the underlying sums 
(of two scores based on halves), i.e., the correlation between the two tests. 

Let us now consider the three-way interaction. It can be shown ([4], pp. 
296-297) that the J X T X M interaction sum of squares reduces to 


(1) 2D (Xia — Xie) — (Kize — Xees)P, 
which is a function of the difference between parallel measures on test 1 and 
on test 2. Cumbersome subscripts may be avoided by setting 
Xia = U, Xin = 2, Xun. = w, 
ta 2. 2a *t,. 2a, ~*s. 
In what follows it must be remembered that uw, v, x and y are in standard 


score form; w and z are not standard scores. 
Expression (1) may be written as 


o Ud u-wvy+ Ve@-y’?-2DVw-ove- yl, 
which becomes 


1(No.-, + No?_, — 2Nr.. + 2Nr., + 2Nr,. — 2Nr,,) 


(3) re = [2(1 a Tur) + 2(1 ay Wiad =r Tus + at oe ye = 2r.¥] 
N 


4 
2 (2 Ses. tag ee + Tuy + ea Tos 
as the value of the J X 7’ X M interaction sum of squares. 

Turning next to the J X T interaction sum of squares, note that by 
assigning to ¢ the explicit values, 1 and 2, the sum of squares may be written as 


2 2 si: asl Som y + 2 > > ies: be ke 
which by a simple procedure ((2], p. 246) reduces to 
p a (Xa. — Xi2.)*, or a (w — 2)” 


in simplified notation. This becomes 


~wt+ 2-2 @, 
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and since w = (u + v)/2 and z = (x + y)/2, 
u+v\ z+) _ (uteyety) 
x ( 2 iZ x ( 2 22> Ne 
=>) vW+ D742. w+ Dri dy 
+2 >. ry —-2 Dur —2 Dw —2 Dox -—2 Dw). 


Each term in the foregoing involves either N times a variance (of unity) or 
N times a correlation coefficient; hence 


(4) x2 + feo + Tey — Tus — Tuy — Tos — Tov) 








is the sum of squares for the J X T interaction. 

The six r’s called for in (3) and (4) can, of course, be computed without 
transforming to standard scores. These r’s are required in Yule’s correction 
for attenuation formula ({1], pp. 209-210). 

The value of r,,, , which ordinarily will be needed for descriptive purposes, 
can readily be obtained in terms of these six r’s by substituting in the appropri- 
ate formula for the correlation of sums (or averages): 


(5) pe lu te Foe Fe, 
V2 ty V2 $F rey 


If the parallel halves of test 1 (also test 2) yield strictly comparable raw 
scores, i.e., equal sigmas, (5) will give a value exactly equivalent to the 
correlation between the raw scores of the two tests. 

When (3) and (4) are divided by their df’s, the two resulting variance 
estimates lead to 








Re Le Pug ln Vi, > Vue 
— ’ 
2 Teg: tay Ce + Tuy + Vox — Tey 





(6) PF, 


with n, = n, = N — 1 as the df’s. 

Since no assumptions were made in arriving at (6), the only assumptions 
to be met are those which underlie the analysis of variance technique. For 
the given situation the assumptions are that the errors of measurement 
entering into the X;,,, scores are independently and normally distributed 
with equal variance. Inequality of variances will occur when the two tests 
differ as to reliability, hence a valid interpretation of F, holds only when the 
reliabilities are the same (or nearly so). These same assumptions hold for 
F, given below. 

A little consideration of the J X M term in Table 2 leads to the a priori 
assumption that it will be zero within chance limits, hence its sum of squares 
may be combined with that for the J X T X M term. Note that this pooled 
sum of squares is nothing more than the residual after taking out the variation 
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due to individuals and that due to J X T interaction. This residual sum of 
squares may be written as 


> > > (Xitm 7 b Pe 


When ¢ is given the explicit values, 1 and 2, and m the values, a and b, 
this sum of squares may be expressed, in simplified notation, as 


Lu w+ DVo-w?+ Ve-y’+ Dw-sa. 

Replacing w by (u + v)/2 and z by (« + y)/2, this can be simplified to 

4 Du-y? +4 D@e-y’, 
which, since the data are in standard scores, becomes 

NQ = 1.) + NUL — 1,,), 
or 
NZ — tus — Foe) 

as the residual sum of squares. The df for this residual is 2V — 2, hence 
(7) F, = 2 Pu + ry > ss = = Tez — Toy 


with n, = N — 1 and n, = 2N — 2 as the df’s. 

It is of interest to see what happens to F, (and indirectly to F,) under 
certain conditions. When measures u and »v are strictly parallel and measures 
x and y are also strictly parallel, it will be seen that the last four r’s in the 
denominator of F, will be identical, thus drop out, and leave a denominator 
equal to that of F, . By utilizing (5), which holds exactly when the measures 
are strictly parallel, the last four r’s in the numerators of (6) and (7) can be 
replaced by 





? 





Twe V(2 + 2r..)(2 + Maid « 
This leads to 





2A tas + toy = toe VE + Br(2 + Brey) 


(8) F; 2 oF Tus re ley 





When the two tests are equally reliable, and r,, = r., = 1, 


= 2 + 2r — 1r,,.(2 + 2r) 


Pe 2—2r ‘ 





or 





oe (i + r)(1 eg om, 
(9) F, ve io % . 
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The r in (9) is the reliability for scores based on half a test. By the Brown- 
Spearman formula the reliability, r’, of total scores would be 


r’ = 2r/1 +n), 
from which 

r=r/(2—-r’). 
Substituting in (9) 
_ Lt r'/2 — ry — re) 





i t—-r7@-ry 
which simplifies to 
1 eat we 
(10) F, = Tae 


Now, finally, if the w and z sets of scores are thought of as having been 

transformed to standard score form, the last expression becomes 

1—r,, 
(11) PF eC: a” ae ’ 
in which o? is the error of measurement variance common to the two tests 
when both the w and the z scores have been transformed to standard scores 
and it is assumed that the two tests have equal reliabilities. 

It is not argued that sample values of F; , F, , and F; will follow the F 
distribution exactly. From (11) it is seen that F, , and by inference F, and 
F, , is a function of the extent to which an observed correlation deviates 
from unity relative to error of measurement variance. 

In applications the Ff, given by (6) or the F, of (7), or their equivalents 
in terms of variance estimates from Table 2, would be used to determine 
whether the correlation between two variables was farther below unity than 
attributable to measurement error. Since F, involves a larger df for the 
error term, it is to be preferred over F, . 


Addenda 


After the foregoing was submitted for publication, Lord [3] presented a 
mathematically elegant but computationally complicated approach to the 
same problem. His likelihood ratio test and the above proposed F test both 
assume independence of measurement errors. His assumption of equality of 
population variances for parallel measures is herein obviated by the trans- 
formation to standard scores. His assumption of the equality of certain 
population covariances, set forth in his equation (2), is equivalent to the a 
priori assumption of no J X M interaction, which, however, requires that 
certain population correlations, instead of covariances, be equal. The F 
method assumes that the two tests are equally reliable (in the population) 
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whereas the likelihood ratio test is not thus restricted. Lord’s is a large 
sample significance test whereas the F test may be used for either large or 
small samples. 

Until such time as the mathematical interrelation, if any, of the two 
solutions has been ascertained, it is of interest to see whether or not the F 
test yields levels of significance similar to those of the likelihood ratio test 
when applied to Lord’s illustrative examples. For this purpose the probabilities 
for his normal deviates have been determined to two figures beyond the cipher, 
and a curve relating p to F has been used for graphical interpolation in order 
to specify the p’s associated with the observed F’s as computed by formula 
(7) above. For the seven examples in Lord’s Table 1, each of which involves 
equal observed reliabilities, we have the following levels of significance 
(Lord’s p’s given first in each pair): .024, .023; .0028, .0025; .17, .17; .062, 
.061; .024, .023; .0062, .0080; .0029, .0025. Admittedly, the graphic determi- 
nations of the fourth decimal figures are rough approximations. 

Lord also applied his method to data on 649 cases for which the observed 
reliabilities were different, .669 and .757. Since the F for these data goes far 
beyond available tabled values, it was transformed to a normal deviate by 
an approximation method given in Wallis and Roberts (({5], p. 458). This 
yielded a normal deviate of 6.19, compared to the 5.94 obtained by the 
likelihood ratio test. For these same data treated as though based on N = 101, 
Lord finds significance at the .010 level whereas the F test leads to .007 by 
both graphical interpolation and the Wallis-Roberts transformation. 

Such close agreement in results as shown in these nine examples may 
not occur when the equal reliability assumption does not hold, but since the 
study by Norton, as reported in Lindquist [2], shows that marked hetero- 
geneity of variances (25, 100, and 225) has little effect on F tests (i.e., a p 
of .01 is near the .02 level and a p of .05 is near the .07 level), it can be pre- 
sumed that such differences in reliabilities as are apt to be encountered in 
practice will not seriously disrupt the F test proposed above. Reliabilities of 
.80 and .95, .60 and .90, .40 and .85, and even .20 and .80, each set of which 
leads to error variances differing by a factor of four, would not be nearly as 
extreme as the differences in the Norton study. 
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THE KUDER-RICHARDSON FORMULA (21) 
AS A SPLIT-HALF COEFFICIENT, AND SOME REMARKS 
ON ITS BASIC ASSUMPTION 


SAMUEL B. LYERLY 
WASHINGTON, D. C. 


Case IV of the Kuder-Richardson series, their formula (21), is derived 
as a generalized — Spearman-Brown coefficient. The basic assump- 
tion employed is shown to be sufficient to justify the various assumptions 
used in derivations by other authors. Some of the implications of this assump- 
tion are discussed. 


Cronbach [1] showed that the Case III reliability coefficient of Kuder 
and Richardson [2] is the mean of all possible split-half Spearman-Brown 
coefficients. This demonstration was useful to students of test theory in 
bringing the Kuder-Richardson formulation closer into line with the familiar 
split-half concept. It is the purpose of this paper to present a development 
of the Kuder-Richardson formula (21) from the split-half point of view 
and to comment on the basic assumption underlying this useful coefficient. 

A test of n items (n assumed to be an even number for convenience) 
may be split into halves in n!/[2(n/2)!’] possible ways; with random splitting 
each possibility may be regarded as equally likely. Suppose all of these 
splits are made for a given individual and all of the possible pairs of half 
tests for him are scored. From the well-known theorem concerning samples 
drawn with replacement from a finite population, the variance of either half- 
score distribution is 

X(n — X;) 
(1) var = a a 
Since the correlation between all pairs of half scores (or any sample of them) 
is — 1, their covariance is 


eager Xn — Xi) 
(2) cov = aD 


Since the pair of scores resulting from one split is as likely as the pair 
resulting from any other split, there is no reason for preferring one pair 
rather than another, so consider the entire set and record them all in a 
bivariate table or plot them all on a scatter diagram. Now repeat this pro- 
cedure for each of a sample of N individuals and compute the product moment 
correlation coefficient over all N sets of pairs of half scores. 

When a number of equally numerous distributions are merged, the 
variance of the resulting total distribution is equal to the average of the 
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variances of the separate distributions plus the variance of their means. 
Using this identity and (1), 


@) aT ao 


(All summations in this and following equations are over the N subjects.) 

Similarly, when equally numerous bivariate distributions are combined, 
the total covariance is equal to the average of the separate covariances plus 
the covariance of the means. In our case each separate bivariate distribution 
(pairs of half scores for an individual subject) is symmetric about a mean of 
(X,;/2, X;/2) and the correlation over all N sets of means must be + 1. 
Their covariance is therefore })X°/4N — (}.X)?/4N’, and thus 


~ 2/22 _ Qn _ 2Xe— HD). 





(4) succes x Gai 4N 4(n — 1) 


Since (3) is the variance of either set of half scores, the correlation 
coefficient is"simply (4) divided by (3). Performing this division gives 
»— 2X? = (2 XIN - YX — Xm — 1) 
DX —-(LX/N + DL XM — X)/m — 1) 
_ Nei + (XOX? —n YF X)/m — 1) 
Ns, ~(2,X ~a 2, Dia) ’ 
where s? is the variance of the total test scores over the N subjects. 
Applying the Spearman-Brown formula, r = 2r’/(1 +r’), and simplifying, 


_&+(2 xX’ =n 2 X)/N@ - 1) 





r 











r 
8, , 
Fe n> X— > Xx’ 
©) a oo le 


which is the Kuder-Richardson formula (21). Replacing )>X by NM, and 
> X* by Ns? + NM?, where M, is the mean of the scores, (5) becomes 


nM, — s; — M: 





r=] — 





(n — 1)s; ‘ 
which simplifies to 
; n M, — Mile] 
(6) r= 8. 


Equation (6) is one of the familiar formulas for computing K-R (21) using the 
mean and variance of the scores and the number of items in the test. 

A number of derivations of K-R (21) have appeared in the literature, 
each using somewhat different sets of stated assumptions. A specific assump- 
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tion used in this paper is implied in the procedure described above in which 
all the possible pairs of half scores for all individuals in the sample were used 
to compute the reliability coefficient. It is assumed not only that a pair of 
scores resulting from one split is as likely to occur as a pair resulting from 
another split for a given subject, but that each such pair can occur in combi- 
nation with any possible pair for any other subject in the sample. This suggests 
one way of stating the difference between the Kuder-Richardson formulas 
(20) and (21). From the split-half point of view, K-R (20) is the expected 
value of the coefficient when the same split is made for every subject. Making 
a different random split for each subject yields a coefficient which converges 
toward K-R (21). 

The above discussion has been couched in split-half terminology. Other 
writers on K-R (21) have stated their assumptions in terms of their own 
approaches to the reliability concept. One basic assumption is required; 
it can be shown that the assumptions in the various derivations, including 
Kuder and Richardson’s original presentation, can be deduced from it. 

The basic assumption for K-R (21) can be stated thus: Jn a test of n 
items which are scored 0 or 1 for failure or success, respectively, each attempt 
at an item by a subject is an independent trial in the Bernoulli sense. A Bernoulli 
trial is an event which has two possible outcomes, which may be called 
success or failure. If a number of independent trials are performed, they 
are indistinguishable so far as probability of success is concerned. Tossing 
coins, throwing dice, and drawing balls from urns are classic examples. 

For mental tests, this assumption implies that individual items have no 
separate identities, even though each may be worded differently or may ask 
a question different from that asked by any other item. Furthermore, item 
equivalence from person to person does not exist. The same item, worded 
identically, may appear on everyone’s test booklet, but item 1 for Subject 
A is regarded as being no more like item 1 for Subject B than it is like item 
2 for Subject B. The two sets of items are, in effect, different samples of 
items from a common pool or population. Since items do not have individual 
identities, any method of splitting the test into halves is necessarily random 
from subject to subject, and the assumption employed in the above split- 
half derivation follows. For the same reason, any item statistic (difficulty, 
variance, item-test correlation, etc.) can be computed only by selecting 
responses from subject to subject at random. These considerations dictate 
the “equal difficulty, equal correlations, matrix of unit rank” assumptions 
of Kuder and Richardson. Lord’s derivation [3], from the point of view of 
parallel tests constructed by random sampling from an item pool, implicitly 
uses this assumption. The K-R (20) formula, on the other hand, “recognizes” 
items and makes use use of item-subject interaction, which is assumed not 
to exist in the K-R (21) formulation. 
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THE AVERAGE SPEARMAN RANK CRITERION CORRELATION 
WHEN TIES ARE PRESENT 


Epwarp E. Cureton 
UNIVERSITY OF TENNESSEE 
This note presents the average Spearman rank correlation between m 
independent rankings and an untied criterion ranking, corrected for ties in 
any or all of the independent rankings. 
Lyerly [2] has considered the average Spearman rank correlation between 
m experimentally independent rankings of 7 persons or objects and a criterion 
ranking, with no ties in any of the rankings, and has shown that its distri- 
bution approaches normality rapidly as m and n increase. 
Let y represent a criterion rank, z a rank in any one of the independent 


rankings, and 
(1) X= Diz. 
Then X is the sum of the independent rankings of one person or object, i.e., 


the sum of the independent rankings for fixed y. With this notation, Lyerly’s 
formula may be written, 


4n +2 , 12 3) Xy 
n—-1  m(n* —n) 





(2) Pa = 1 


The total sum of squares of differences between a y and an 2, as given by 
Lyerly, is 


(3) - a 2mn(n + ae i. 9 > Xy. 


Dividing by m to obtain the average and then substituting this average 
for >> d’ in the usual formula for-the Spearman rank correlation 
[o = 1 — 6 >. d’/(n*® — n)], the result reduces to (2). 

Now assume that ties may be present in any or all of the independent 
rankings. When any one such independent ranking is correlated with the 
criterion ranking, Kendall ({1], p. 31) shows that >> d? must be increased 
by >> (#8 — #)/12, where ¢ is the number of persons or objects having equal 
ranks in each tied set, and the summation is over all sets of ties in the in- 
dependent ranking. Let 
(4) T= DL (- 9, 
the summation now. being over all sets of tied ranks in all m independent 
rankings. Then (3) must be increased by adding 7'/12 to the terms on the 
right. Dividing by m to obtain the average value of >> d’ corrected for ties, 

271 











272 PSYCHOMETRIKA 


and substituting the resulting expression for >> d’ in the formula for the 

Spearman rank correlation, yields 

_4n +2, 24 DO Xy-T 
n—1 2m(n* — n) 





(5) Pa = 1 


’ 


the average Spearman rank criterion correlation corrected for ties. 

Note that p,, as defined by (5) does not have limits + 1 when ties are 
present; the ties in any one independent ranking prevent complete agreement 
with the criterion ranking even if the order of its ranks is otherwise identical 
with that of the criterion series. This formula is in fact a generalization of 
Kendall’s p, ({1], p. 29). The criterion ranking, moreover, may not contain 
ties; the problem of the correlation between an independent ranking and a 
criterion ranking which includes true ties has never to the writer’s knowledge 
been investigated. 

Kendall shows ({1], p. 64, footnote) that the standard error of p in the 
null case is the same when ties are present as when they are not. Hence the 
critical ratio, 


(6) CR = pa Vmn — 1), 


may be taken as a unit-normal deviate in testing the hypothesis that the 
true average correlation is 0, whether or not ties are present. There is no 
corresponding theory to cover the non-null case. 

The approach of the distribution of p,, to normality with increasing n 
and mm is fairly rapid in the untied case, and should not be appreciably slower 
when ties are present except in extreme cases. A conservative recommen- 
dation would be to use (6) whenever: 


(a) n> 5andm+n > 20; 
(b) not more than 2/3 of all the ranks in the independent rankings are 
tied; 

(c) every independent ranking has at least three distinct values. 
When these conditions are met, no correction for continuity is needed; this 
is fortunate, as the proper correction would be difficult to determine. When 
the above conditions are not met, there is no good significance test. The 
applicability of the ¢ approximation to p,, has not been demonstrated, and 
direct computation of the exact probability would be prohibitively time 
consuming even for small m and n when ties are present. 
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NOTE ON “EFFICIENT ESTIMATION AND 
LOCAL IDENTIFICATION IN LATENT CLASS ANALYSIS’* 


Ricuarp B. McHucu 
UNIVERSITY OF MINNESOTA 


I am indebted to Dr. Albert Madansky for detecting the following two 
errata and for suggesting the corrections outlined below. . 

(1) The information functions were incorrectly given ({1], p. 346). They 
may be corrected as follows: 

Write “numerator of @(¢)(isis++i>) AS Qe) irsa---sp) ANG “numerator of 
QsserCisiarrép)” BS Qeisey éréaeeetp) - Phen 





A A 
1; a b> [cor cirtarerigy — Qerrtirie-srin UM Qeerr irieerriny — Gey tiste--tpy] 
(e)(e’) = 
n D 


Jiriarrriy 
fore,c’ = 1,2,---,y¥—- 1; 


A A 
1 Vise) Ciriase tip) Qi’ se") Cisioe* tip) Jise) Ji’ se’) 


-] ise) (i’se’) = 
n RS SING! 302 > Jisiarseig(Jcese) ae oa a 6a) (cir se") —s 55-4) 


for 1,7’ = 1,2, ---,p,¢,ce = 1,2, >>> , 7; and 








A A A 
} [Giery Ciriarseip) <a Gey Cintas seip) Geiser risen) JUize) 


1, sn 
nm (0 ése) fond, Jiriaee*ép Qesse) —1+ 5.2) 


force’ = 1,2,---,y—1,c = 1,2, ---,7, and 1 = 1, 2, --- , p, where 
i 1 if question 7 is answered yes in the dth member of D, 
a 
0 otherwise. 


The errors arise in differentiating log LZ with respect to f, and g;:. . 
When differentiating log L with respect to f, ,c = 1, --- ,y — 1, one must 
differentiate f, = 1 — f: — fe — -++: —fy-1 with respect to f, as well; when 
differentiating log L with respect to g;;. , one must evaluate 


0 [fe II 9 ise) I] (1 14 Jci':e)] 


iePa i’ePa 


OI cise) 





for each d, where P; is the set of questions answered “positively” for the dth 
member of D. After taking these points into account in determining S, and 
S;.. , the information functions as stated above are easily obtained. 
* Psychometrika, 1956, 21, 331-347. 
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(2) The statement was also made ({1], p. 337) that if an estimator of 
the structural parameter @ is consistent, then @ must be locally identifiable. 
Actually, if the estimator is consistent an even stronger statement can be 
made, namely that @ is identifiable (see [2], p. 376). However, since the 
proposed estimator of the latent parameters is consistent only when certain 
(local) restrictions and regularity conditions on the parameters are met, 
the fact that the estimator is consistent only implies that the parameters 
are locally identifiable and not identifiable. 
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BOOK REVIEWS 


Henry QuastiER (Editor). Information Theory in Psychology: Problems and Methods. 
Glencoe, Illinois: The Free Press, 1955. Pp. x + 436. 


Several patterns for presenting the proceedings of specialized scientific conferences 
have developed. Differences in patterns can be exemplified in reports on the one topic 
with which the book under review is concerned—information theory. One extreme is 
represented by the book published in 1953 by the Josiah Macy, Jr. Foundation, Cybernetics, 
Circular Causal and Feedback Mechnisms in Biological and Social Systems, which seemed to 
have been a scarcely retouched publication of the working papers as they were actually 
presented in the conference, together with a practically verbatim transcript of all the 
discussion. A report of another conference on information theory held in 1954 was multi- 
lithed and published more or less informally by J. C. R. Licklider of M.I.T.; this consisted 
of a somewhat filtered and digested version of what went on, Licklider using himself as a 
transducer. A more standard pattern was represented by the University of Pittsburgh’s 
Current Trends in Information Theory (1953), presenting simply a series of scientific papers, 
amply edited and proofread by their authors, who were invited to read them at this 
conference. 

The present volume invites favorable comparison with all of the afore-mentioned. 
Nearly all the working papers presented at the conference are printed as edited by their 
authors after the conference, and interwoven is a sort of running comment by the editor, 
giving his own reactions as well as relaying the significant points which came up in the 
discussion at the conference. Furthermore, the papers are arranged not in the order in 
which they were presented at the conference but in an order which the editor found after 
the conference to be more logical and advantageous. 

The editor is able to make the reader feel that he is not missing any of the really 
significant discussion, but he cautions that a number of “really good papers’’ were not 
published because their authors could not find time to prepare them for publication. 

So much for the format of this varityped, offset-printed volume. What of its con- 
tent? Even though it can hardly be said to have the appearance or organization of a text- 
book, the book could well become one of the standard references on information theory 
in psychology, for it represents the first serious attempt to assay the positive gains in this 
field and to standardize notation and terminology, instead of merely presenting a somewhat 
disorganized array of interesting suggestions and speculations, as do some of the other 
works on information theory. 

In this respect, a most important feature is the generalization of information theory 
beyond the bounds of its most frequent application in communication. A few quotations 
from the summary composed by the conference members will be of interest. ‘It is basic to 
information theory that any event is evaluated against the background of the whole class 
of events that could have happened. Information theory proposes to measure the effect 
of operations by which a particular selection is made out of a range of possibilities. ... In 
information theory, variation is the indispensable basis of selection, discrimination, com- 
munication, specification, and related operations. Traditional statistics is largely concerned 
with what can be done or said in spite of variation; information theory deals with what 
can be done because of variation. .. . If two objects or events are related, then they must 
mutually affect each other’s range of possibilities. Thus, relatedness amounts to a mutually 
selective operation. ... In this way, information theory describes the strength of coupling 
between components of a system, parts of a structure, members of an organization, and 
portions of an ordered sequence in time. ... The relation most commonly studied is com- 
munication.” 
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Seen in this light, information theory may provide a way of measuring more ob- 
jectively the dynamic relations existing among the parts of Gestalten. In fact there were 
many hints that the conference members were beginning to be concerned with the study 
of complex dynamic systems, particularly those represented by the human being as a 
transducer of input. 

Perhaps out of modesty, one of the most valuable papers ‘Standardized Nomen- 
clature, An Attempt,’’ was placed at the end of the section on “foundations,” by the 
editor, who was also its author. Users of information theory are put on notice, regardless 
of their personal predilections, that the basic system of notation in information theory 
will involve measures of H (uncertainly or specificity), T (relatedness or communication), 
A (partialled relatedness), D (constraint), and C (redundancy), together with associated 
subscripts, parenthetical terms, and the like. 

Along with notation, one would demand computational procedures, and they are 
all here, even tables of log, n and —p loge p provided by Klemmer from an Air Force 
Technical Report. McGill gives a concise and elegant statement of his well-known work 
on the relation between multivariate analysis and information theory. There are reports 
by Miller, Rogers, Green, and Augenstine of noble attempts to determine the properties 
of sampling distributions of information functions; while the sampling distributions have 
not yet been determined for the general case, Miller’s finding that the basic informational 
function, H, is a biased statistic should receive wide attention. Attneave shows, however, 
that under certain conditions a better estimate of H than that achieved by the Miller- 
Madow correction can be made from averaged values in a transmission matrix. The 
section on information measures as such comes to a close with a chapter by Henry Quastler 
on approximate procedures for estimating information measures. 

By far the largest portion of the book is devoted to reports of psychological experi- 
ments, or to theoretical papers concerning special applications of information theory. In 
line with the earlier insistence that ‘ccommunication”’ is only one area where information 
theory can be applied, there is almost a complete absence of attention to language or to 
any kind of symbolic, conventional communication between persons. The closest one 
comes to finding anything of this nature, in fact, is in Osgood’s paper, which reports work 
on the dimensionality of facial expression as a vehicle for communicating emotion, and in 
a paper by Fritz and Grier on air traffic control communications. On the contrary, most 
of the papers are concerned with “communication,” if it can be called that, between 
human organisms and machines. We find, first, that the activity of the organism itself 
seems to be “‘unitized,’’ timewise, over what Stroud calls ‘‘moments,’”’ almost as if human 
experience existed in a series of frames like those of a motion picture film. 

The very interesting question is raised as to how much information can be contained 
with each moment or frame and this question is further pursued by Hake in a note on 
the concept of channel capacity. Soon we are looking at the organism starting to interact 
with a machine—a machine (or rather, an apparatus) which presents, by means of lights 
or buzzers, something which is in effect a stochastic process; the task of the organism is 
to judge the relative frequency of events or to predict at any point of time what event will 
take place next. Two types of theory, the oné cognitive, the other associationistic, have 
been offered to account for the organism’s disturbingly nonstrategic behavior in this 
situation. Further papers, by Senders and Cohen, Deininger and Fitts, and others, discuss 
the application of information theory to studying the behavior of people reading dials or 
responding to various other types of information displays. A particularly interesting paper 
is that by Bricker, who after a review of reaction time experiments suggests that the 
amount of information processed is linearly related to the time required for processing. 

All in all, this is an excellent series of exercises in quantitative rationalizing of psy- 


chology. 
JouN B. CARROLL 


Harvard University 
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Wusam G,. CocHraN AND GERTRUDE M. Cox. Experimental Designs. New York: John 
Wiley & Sons, Inc. Second Edition 1957. Pp. xiv + 611. 


This excellent revision of a well-known and successful textbook is better described 
as a judicial and timely extension than as a reformulation. Experimental Designs, first 
published in 1950, provided an orderly and much needed practical description of experi- 
mental designs important for biological research, particularly agricultural experimentation. 
Cochran and Cox had sought to prepare a handbook of precedures, and from the arith- 
metical and practical standpoint their Experimental Designs was admirably complete. It 
was not a beginner’s cook book, however, and a working knowledge of the statistics in- 
volved in experimental analysis, particularly analysis of variance, was assumed. 

The present (second) edition, although written in the framework of the original 
edition, incorporates numerous substantial improvements and extensions. The authors’ 
presentation of the various currently useful experimental plans involves both the intro- 
duction of new approaches and the addition of recent developments in well-established 
procedures. In this way they have sought to accommodate their second edition to the 
continuously emerging requirements for experimental designs. Because of changes in the 
present volume and because of the growth of experimentation in industry, this second 
edition will find many more applications in industry than the original work. Nevertheless, 
references to specific industrial problems or processes are relatively few. For the most 
part the references are from the fields of mathematical statistics, biometry, and agri- 
culture. 

This is not a volume in which the typical reader from the behavioral sciences will 
feel at home. Almost all the problems and many of the plans are alien to the experience 
of the behavior scientist, and for this reason he may not recognize plans which are appli- 
cable to his research interest. He will be tempted, moreover, to adapt or modify his re- 
search interests to take advantage of some of the elegance and economy represented by 
many of the plans included in this volume. This is not unusual. Many of the more elegant 
designs employed by psychologists are borrowed from other fields, and certainly designs 
developed in other research areas can not be fairly gauged on the basis of their obvious 
and immediate relevance to the needs of those who investigate behavior. 

Although the present edition is sufficiently concerned with specific procedures and 
explicit cautions to justify its being described as an extremely practical guide to experi- 
mental plans, it is not written at the introductory level. Like its predecessor, this book 
presupposes both sound statistical training and substantial experience in the experimental 
study of various problems. From the standpoint of those who have not fully mastered 
basic statistical analysis, Chapter 3 is particularly and significantly instructive. It provides 
a useful summary of analysis of variance and applications of some of the assumptions 
involved in least squares procedures. 

In addition to the revisions and extensions of the various chapters from the original 
edition, the present volume contains two new chapters. Chapter 6A deals with the use 
of fractional replication in factorial experiments. Chapter 8A includes a section on the 
use of discontinuous data, characteristic of many formal analytical plans, with the inter- 
pretation of various functions as continuous trends, The readers will find many new designs 
in this edition including new incomplete block designs and new incomplete factorial designs. 
The scope of the volume is best indicated by the chapter headings: 1, Introduction; 2, 
Methods for Increasing The Accuracy of Experiments; 3, Notes on the Statistical Analysis 
of the Results; 4, Completely Randomized, Randomized Block, and Latin Square Designs; 
5, Factorial Experiments; 6, Confounding; 6A, Factorial Experiments in Fractional Repli- 
cation; 7, Factorial Experiments with Main Effects Confounded: Split-plot Designs; 8, 
Factorial Experiments Confounded in Quasi-Latin Squares; 8A, Some Methods for the 
Study of Response Surfaces; 9, Incomplete Block Designs; 10, Lattice Designs; 11, Balanced 
and Partially Balanced Incomplete Block Designs; 12, Lattice Squares; 13, Incomplete 
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Latin Squares; 14, Analysis of the Results of a Series of Experiments; 15, Random Permu- 
tations of 9 and 16 Numbers. 

It is possible that some of the designs which are appropriate to sequential experi- 
mentation in industry may find application in the studies of the behavior scientist. Cer- 
tainly the economies promised by the various kinds of confounding plans will continue 
to challenge the psychological investigator. 

This book with its handsome format, most readable type, and well-designed pres- 
entation of analyses and plans will continue to be greatly appreciated as a textbook. The 
fact that the new edition comprises additions, refinements, and improvements without 
changing the structure or the organization of the original volume should make it very 
convenient for the teacher to introduce the new text into courses which used the old text. 
The authors are to be commended for undertaking this revision and for their lucid pres- 
entation of an intricate body of knowledge. This book can be of great value both to those 
who perform experiments and to those who responsibly read experimental reports. 


J. R. W1TTENBORN 
Rutgers University 














